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EXECUTIVE SUMMARY 


In the rapidly changing world of the 21st century, science literacy is an essential goal for 
all of our nation’s youth. Through science education, children come to understand the 
world in which they live and learn to apply scientific principles in many facets of their 
lives. In addition, our country has an obligation to provide young people who choose to 
pursue careers in science and technology with a strong foundation for their postsecondary 
study and work experience. The nation’s future depends on scientifically literate citizens 
who can participate as informed members of society and as a highly skilled scientific 
workforce, well prepared to address challenging issues at the local, national, and global 
levels. Recent studies, including national and international assessments, indicate that our 
schools still do not adequately educate all students in science. 


Science seeks to increase our understanding of the natural world through empirical 
evidence. Such evidence gathered through observation and measurement allows for an 
explanation and prediction of natural phenomena. Hence, a scientifically literate person is 
familiar with the natural world and understands key facts, concepts, principles, laws, and 
theories of science, such as the motion of objects, the function of cells in living organ- 
isms, and the properties of Earth materials. Further, a scientifically literate person can 
connect ideas across disciplines; for example, the conservation of energy in physical, life, 
Earth, and space systems. Scientific literacy also encompasses understanding the use of 
scientific principles and ways of thinking to advance our knowledge of the natural world 
as well as the use of science to solve problems in real-world contexts, which this docu- 
ment refers to as “Using Technological Design.” 


The National Assessment of Educational Progress (NAEP) and its reports are a key 
measure in informing the nation on how well the goal of scientific literacy for all students 
is being met. The Science Framework for the 2015 National Assessment of Educational 
Progress sets forth the design of the NAEP Science Assessment. The 2015 NAEP Sci- 
ence Assessment will use the same framework used in 2009. The 2009 NAEP Science 
Assessment started anew NAEP science trend (i.e., measure of student progress in sci- 
ence), and the 2015 NAEP Science Report Card will include student performance trends 
from 2009 to 2015. Trends in student science achievement were reported from 1996 to 
2005 as well. However, the trend from 1996 to 2005 was not continued due to major dif- 
ferences between the 2005 and 2009 frameworks. The 2009 — 2015 framework represents 
a unique opportunity to build on key developments in science standards, assessments, and 
research. This document is intended to inform the general public, educators, policymak- 
ers, and others about what students are expected to know and be able to do in science as 
part of The Nation’s Report Card, a program of the U.S. Department of Education (ED) 
that reports on NAEP findings. 


In 1988, Congress created the National Assessment Governing Board to set policy for 
NAEP. The National Center for Education Statistics (NCES, a division of ED) carries out 
NAEP. As the ongoing national indicator of the academic achievement of U.S. students, 
NAEP regularly collects information on representative samples of students in grades 4, 8, 
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and 12 and periodically reports on student achievement in reading, mathematics, writing, 
science, and other subject areas. NAEP scores are always reported at the aggregate level, 
not for individual students or schools. (By law, NAEP cannot report results for individual 
students.) For science, NAEP results are reported at the national and state levels and for a 
number of large urban districts. The district reports are provided for urban school systems 
that volunteer for the Trial Urban District Assessment component of NAEP. 


NAEP produces comparative student achievement results according to demographic 
factors such as gender, race/ethnicity, and geographic region. Results are also provided in 
terms of student, teacher, and school background variables related to science achieve- 
ment. Taken together, this information from NAEP helps the general public, educators, 
and policymakers make informed decisions about education. Interested individuals can 
access performance results and released questions through NAEP reports and websites. 


Beginning in 2009 a new framework to guide the science assessment was necessary for 
several reasons: publication of National Standards for science literacy, advances in both 
science and cognitive research, growth in national and international science assessments, 
and increases in innovative assessment approaches. This framework presents the content 
to be assessed as well as the conceptual base for the assessment. It is intended for a gen- 
eral audience. A more detailed technical document, Science Assessment and Item Specifi- 
cations for NAEP, is a companion piece. The specifications document provides detailed 
information on the content to be assessed, item development, and other aspects of as- 
sessment development and administration. The audience for the science specifications is 
NCES and the NAEP assessment development contractors. 


KEY FEATURES 


This framework is the result of extraordinary effort and commitment by hundreds of 
individuals across the country, including some of the nation’s leading scientists, science 
educators, policymakers, and assessment experts. Under contract to the Governing Board, 
WestEd and the Council of Chief State School Officers (CCSSO) spent 18 months devel- 
oping the framework; this process involved committees, regional hearings, and other 
public forums. The Governing Board also engaged an external review panel to evaluate 
the draft framework and convened a public hearing to gather additional input during the 
development process. 


The 2009 — 2015 framework incorporates the following key features: 


e Its design is based on widely accepted national science standards and assessments 
in addition to state curriculum standards. However, it is intended to inform devel- 
opment of an assessment, not to advocate for a particular approach to instruction 
or to represent the entire range of science content and skills. 


e In view of the need to keep the United States and its youth internationally compet- 
itive in science and technology, the framework development process gave special 
consideration to international assessment frameworks, such as those for Trends in 


NAEP 2015 SCIENCE FRAMEWORK 


ix 


International Mathematics and Science Study (TIMSS) and Programme for Inter- 
national Student Assessment (PISA). 


e The breadth of the science principles represented in the source materials made it 
necessary to focus on the foundational and pervasive knowledge within each dis- 
cipline and to pare down the science content to be assessed. 


e The framework is based on scientific knowledge and processes derived from test- 
ed explanations and supported by accumulated empirical evidence. Explanations 
of natural phenomena that rely on nonscientific views are not reflected in the 
framework. 


e Science content is presented in detailed, grade-specific charts that also allow the 
reader to see the progression in complexity of ideas across grades. 


e Every attempt has been made to be free of error in describing the science content. 
The language used strives to be accurate but not technical so as to make the 
framework accessible to a wide audience. 


e The focus is on students’ conceptual understanding, that is, their knowledge and 
use of science facts, concepts, principles, laws, and theories. Students’ abilities to 
engage in some components of scientific inquiry and technological design are also 
reflected in the framework. 


e New types of items are recommended, including the use of interactive computer 
tasks. 


CONTENT OF THE FRAMEWORK 


The framework describes the science content and science practices that form the basis for 
the NAEP Science Assessment. It also discusses item distribution and types of items, as 
well as draft achievement levels. Finally, it recommends several small-scale, special 
studies to be considered in conjunction with the science assessments. 


SCIENCE CONTENT 


The science content for NAEP is defined by a series of statements that describe key facts, 
concepts, principles, laws, and theories in three broad areas: 


e Physical Science 
e Life Science 
e Earth and Space Sciences 


Physical Science deals with matter, energy, and motion; Life Science deals with struc- 
tures and functions of living systems and changes in living systems; and Earth and Space 
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Sciences deal with Earth in space and time, Earth structures, and Earth systems. Details 
about the science content and the science content statements can be found in chapter two. 


SCIENCE PRACTICES 


The second dimension of the framework is defined by four science practices: 


Identifying Science Principles 
Using Science Principles 
Using Scientific Inquiry 
Using Technological Design 


These practices can be combined with any science content statement to generate student 
performance expectations, and assessment items can then be developed based on these 
performance expectations. The cognitive demands placed on students as they engage in 
assessment tasks are also described. The science practices and cognitive demands are 
explained more fully in chapter three. 


DISTRIBUTION OF ITEMS 


As measured by student response time, the distribution of items by content area should be 
as follows: approximately equal across Physical Science, Life Science, and Earth and 
Space Sciences at grade 4; more emphasis on Earth and Space Sciences at grade 8; anda 
shift to more emphasis on Physical Science and Life Science at grade 12. With respect to 
science practices, at all grades the greatest emphasis should be on identifying and using 
science principles and slightly less than a third of the time should be spent on items 
related to using scientific inquiry. Specific recommended percentages are discussed in 
chapter four. 


TYPES OF ITEMS 


Item types for the NAEP Science Assessment fall into two broad categories: selected- 
response items (such as multiple choice) and constructed-response items (such as short 
answer). As measured by student response time, 50 percent of the assessment items at 
each grade level should be selected response and 50 percent should be constructed 
response. To further probe students’ abilities to combine their understanding with the 
investigative skills that reflect practices, a subset of the students sampled should receive 
an additional 30 minutes to complete hands-on performance or interactive computer 
tasks. At each grade, at least four of these tasks should be included. Of these four tasks, 
there should be at least one hands-on performance task and one interactive computer 
task; the number of interactive computer tasks should not exceed the number of hands-on 
performance tasks. Chapter four contains more information about types of items. 
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Hands-on Performance Tasks 


In hands-on performance tasks, students manipulate selected physical objects and try to 
solve a scientific problem involving the objects. NAEP hands-on performance tasks 
should provide students with a concrete task (problem) along with equipment and 
materials. Students should be given the opportunity to determine scientifically justifiable 
procedures for arriving at a solution. Students’ scores should be based on both the solu- 
tion and the procedures created for carrying out the investigation. Further discussion 
about hands-on performance tasks can be found in chapter four. 


Interactive Computer Tasks 


There are four types of interactive computer tasks: (1) information search and analysis, 
(2) empirical investigation, (3) simulation, and (4) concept maps. Information search and 
analysis items pose a scientific problem and ask students to query an information data- 
base and analyze relevant data to address the problem. Empirical investigation items 
place hands-on performance tasks on the computer and invite students to design and con- 
duct a study to draw conclusions about a problem. Simulation items model systems (e.g., 
food webs) and ask students to manipulate variables, and predict and explain resulting 
changes in the system. Concept map items probe aspects of the structure or organization 
of students’ scientific knowledge by providing concept terms and having students create a 
logical graphic organizer. Chapter four contains more information about interactive com- 
puter tasks. 


SPECIAL STUDIES 


On occasion, NAEP carries out special studies to inform future assessments. Three 
special study proposals ranked as top priority for the NAEP Science Assessment: 


e “Exchangeability” of hands-on performance and interactive computer investiga- 
tions 


e Impact of variation in item format and language demand on the performance of 
English language learners and students with disabilities 


e Computer adaptive testing to assess the development of student understanding of 
Earth systems 


Details of these proposed studies are found in chapter four. 


ACHIEVEMENT LEVELS 


Results of the NAEP Science Assessment are reported as average scores for groups 

of students and as percentages of students who attain the Basic, Proficient, or Advanced 
achievement levels. Descriptions of these achievement levels are contained in 

appendix B. 
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NAEP Science Project Staff and Committees 


The Science Framework for the National Assessment of Educational Progress is the 
result of extraordinary effort and commitment by hundreds of individuals across the 
country, including some of the nation’s leading scientists, science educators, policy- 
makers, and assessment experts. Every attempt has been made to be accurate in the 
description and representation of the science content. Existing framework development 
policies and procedures ensure periodic reviews and revisions, if necessary, to reflect 
advancements in scientific knowledge. Because this is a public endeavor, the Governing 
Board (www.nagb.org) welcomes suggestions for future versions of the framework. 
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CHAPTER ONE 
OVERVIEW 


NAEP measures student science achievement nationally, state by state, and, most 
recently, across selected urban school districts. Periodically, the framework underlying 
the science assessment is revised or updated. This document, Science Framework for the 
2015 National Assessment of Educational Progress, is based on the same framework 
used for the 2009 NAEP Science Assessment. The framework provides guidance on the 
science content to be assessed, the types of assessment questions, and the administration 
of the assessment. 


For more than 35 years, NAEP has gathered information on student achievement in 
selected academic subjects. Originally, assessments were age-based samples of 9-, 13-, 
and 17-year-old students. Beginning in 1983, the assessment also has included grade- 
based samples of students in grades 4, 8, and 12. Currently, long-term trend NAEP con- 
tinues to assess 9-, 13-, and 17-year-olds in mathematics and reading, while main NAEP 
assesses students in grades 4, 8, and 12. More information about differences between 
long-term trend and main NAEP can be found on the Internet at 
http://nces.ed.gov/nationsreportcard/about/Itt_main_diff.asp (NCES 2005b). 


NAEP has become an important source of information on what U.S. students know and 
are able to do in reading, mathematics, science, U.S. history, writing, and other subjects. 
In addition, NAEP provides information on how student performance has changed over 
time. Since the 1990s, in addition to the national-level assessments, NAEP has conducted 
and reported state-level assessments at grades 4 and 8 in reading, mathematics, writing, 
and science. State-level as well as national science assessments were conducted in 1996, 
2000, and 2005. The resulting data on student knowledge and performance have been 
accompanied by background information that allows analyses of a number of student de- 
mographic and instructional factors related to achievement. The assessments have been 
designed to allow comparisons of student performance over time and among subgroups 
of students according to region, parental education, gender, and race/ethnicity. In 2002, 
NAEP began a Trial Urban District Assessment (TUDA) in districts that volunteered to 
participate. TUDA has continued through 2005, when 10 districts took part in NAEP 
assessments that produced district-level results. The TUDA program has grown in size, 
with 21 districts volunteering to participate in 2015. 


NEED FOR A NEW FRAMEWORK 


The framework that guided the NAEP Science Assessments administered in 1996, 2000, 
and 2005 was developed in the early 1990s. Since then, the following developments have 
taken place, making it necessary to create a new framework for assessing science in 2009 
and beyond: 
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e Publication (for the first time) of National Standards for science literacy in 
National Science Education Standards (National Research Council [NRC] 1996) 
(National Standards) and Benchmarks for Scientific Literacy (American Associa- 
tion for the Advancement of Science [AAAS] 1993) (Benchmarks). Since their 
publication, these two national documents have informed state science standards. 


e Advances in science research (e.g., in the relationship between human activity 
and the natural world) that have increased knowledge and, as a consequence, have 
influenced the school curriculum in the fields of physical, life, and Earth and 
Space Sciences. 


e Advances in cognitive research (e.g., in how students learn increasingly com- 
plex material over time) that have yielded new insights into how and what 
students learn about science (NRC 1999c). 


e Growth in the prevalence of science assessments nationally and internation- 
ally, including the requirements in the current federal education legislation (No 
Child Left Behind) for science assessment starting in 2007, the ongoing interna- 
tional assessment (TIMSS) (http://timss.bc.edu), and PISA (www.pisa.oecd.org). 


e Growth in innovative assessment approaches that probe students’ understand- 
ing of science in greater depth than before (e.g., clusters of items tapping 
students’ conceptions of the natural world), sometimes with the use of computer 
technology (NRC 2001). 


e Increased inclusion of formerly excluded groups in science assessments (e.g., 
students with disabilities and English language learners), requiring a new assess- 
ment to be as accessible as possible and also to incorporate accommodations so 
the widest possible range of students can be fairly assessed. Accommodations 
should not alter the science constructs being measured. 


CONTEXT FOR PLANNING THE FRAMEWORK 


Any NAEP framework must be guided by NAEP purposes as well as the policies and 
procedures of the Governing Board, which oversees NAEP. For the NAEP Science 
Assessment, the main purpose of the framework is to establish what students should 
know and be able to do in science. Meeting this purpose requires a framework built 
around what communities involved in science and science education consider as a rigor- 
ous body of science knowledge and skills that are most important for NAEP to assess. 


In prioritizing the content, the framework developers used the NAEP Science Assessment 
steering committee guidelines (presented later in this chapter), which recommended the 
two national documents (National Standards and Benchmarks) as representative of the 
leading science communities and their expectations for what students should know and be 
able to do in science. As curriculum frameworks, however, these documents cover a very 
wide range of science content and performance. The inclusive nature of both documents 
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demonstrates the difficulty of identifying a key body of knowledge for students to learn 
in science and, therefore, what should be assessed. Neither document limits or prioritizes 
content as is necessary for developing an assessment, posing a considerable challenge to 
the framework developers. The development of the framework also was informed by 
research in science and science education, best practices, international assessment 
frameworks, and state standards. 


THE FRAMEWORK DEVELOPMENT PROCESS 


In September 2004, the Governing Board awarded a contract to WestEd and CCSSO to 
develop a recommended framework and specifications. WestEd and CCSSO, in collabo- 
ration with AAAS, the Council of State Science Supervisors (CSSS), and the National 
Science Teachers Association (NSTA), used a process designed to accomplish the pur- 
poses of this project with special attention given to the assessment issues specific to K-12 
science achievement. The process for developing the framework and related products was 
inclusive and deliberate, designed to achieve as much broad-based input as possible. 


A two-tiered committee structure, consisting of a steering committee and a planning 
committee, provided the expertise to develop the framework as specified by the Govern- 
ing Board. (See “NAEP Science Project Staff and Committees” for lists of committee 
members.) The two committees were composed of members who were diverse in terms 
of role, gender, race/ethnicity, region of the country, perspective, and expertise regarding 
the content of the assessment to be developed. 


The science assessment steering committee included leaders in science, science educa- 
tion, general education, assessment, and various public constituencies and set the course 
for the project. Functioning as a policy and oversight body, this group developed a charge 
that outlined the planning committee’s responsibilities in developing the framework. The 
committee also reviewed and provided feedback on drafts of the framework and related 
materials. 


The science assessment planning committee, supported by the project staff, was the 
development and production group responsible for drafting the framework, the specifica- 
tions, recommendations for background variables, designs for one or more small-scale 
studies, and preliminary science achievement level descriptions. This committee was 
made up of science teachers, district and state science personnel, science educators in 
higher education, scientists, and assessment experts. The planning committee’s work was 
guided by policies, goals, and principles identified by the steering committee. In addition, 
the planning committee used a number of resources to facilitate its work. These resources 
included an issues and recommendations paper (Champagne et al. 2004) developed spe- 
cifically for this NAEP project; the frameworks and specifications for the 2005 NAEP 
Mathematics Assessment and 1996-2005 NAEP Science Assessments; other NAEP 
reports and documents produced by the Governing Board and NCES; international 
assessment frameworks; syntheses of state and national curriculum standards; and 
research papers and resources provided by steering and planning committee members and 
project staff. 
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The structure for conducting the work consisted of a series of meetings. From December 
2004 through September 2005, the steering committee met three times and the planning 
committee met six times; two of the steering committee meetings overlapped with plan- 
ning committee meetings. Governing Board staff supported and participated in the work 
of the committees during the meetings. In addition, between formal work sessions, Gov- 
ering Board members and staff provided ongoing feedback and guidance on project 
documents and processes. 


During spring 2005, CCSSO led a series of outreach efforts to solicit feedback on draft 
versions of the framework. Formal activities included the following: 


e A series of 13 regional meetings held across the country and hosted by CCSSO 
and CSSS members 


e A national meeting of CSSS representatives 
e A Web-based survey of science teachers distributed through NSTA 


e An invitational science and industry feedback forum held in Atlanta, Georgia, in 
conjunction with a Governing Board meeting 


These activities are discussed in A Summary of National Feedback Provided on Prelimi- 
nary Drafts Gathered from Surveys and Regional and National Feedback Meetings 
(CCSSO 2005). Feedback from these sessions has been incorporated into the framework. 
Examples include reducing the number of statements of science content to be assessed; 
comparing the old and new science frameworks; and ensuring a high level of consistency 
in scope, specificity, language, and format across the science content areas. 


Other related outreach activities included but were not limited to presentations and 
sessions held with AAAS, the CSSS annual conference; the NSTA national and regional 
conventions; meetings of the NRC’s Board on Science Education and Committee on 
Science Learning K—-8; and CCSSO’s Mega-SCASS (State Collaborative on Assessment 
and Student Standards) conference, large-scale assessment conference, and Education 
Information Management Advisory Consortium (EIMAC). The Governing Board 
engaged an external review panel to evaluate the draft framework and convened a public 
hearing to gather additional input during the development process. (See “NAEP Science 
Project Staff and Committees” for more complete lists of individuals and organizations 
that contributed to the development of this framework.) The planning committee 
reviewed feedback from these groups as well as that from the steering committee and 
made changes as it deemed appropriate. After final approval from the steering committee, 
the framework, specifications, and related products were submitted to the Governing 
Board for approval. The Governing Board unanimously approved the Science Framework 
on November 18, 2005. Final copies of the specifications and related products were sub- 
mitted to the Governing Board in March 2006. 
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STEERING COMMITTEE GUIDELINES 


Because the steering committee guidelines were important to the planning committee in 
developing the framework, the major points of the guidelines are summarized below. 
(The complete document is contained in appendix A.) The guidelines consist of criteria 
that the framework and specifications must meet: 


e The framework is informed by the National Standards and Benchmarks. The 
framework should reflect the nation’s best thinking about the importance and age- 
appropriateness of science principles and thus be informed by two national docu- 
ments that were subject to extensive internal and external reviews during their 
development. 


e The framework reflects the nature and practice of science. The National 
Standards and Benchmarks include standards that address science as inquiry, na- 
ture of science, history of science, and the manmade world. The framework 
should emphasize the importance of these aspects of science education and should 
include the expectation that students will understand the nature and practice of 
science. Because the scientific disciplines are no longer practiced in isolation and 
research that cuts across discipline boundaries is common, the framework should 
identify some of the science concepts and skills that cut across the assessed 
content areas. The framework should address science in both the natural and 
manmade world, as well as social and historical contexts. 


e The framework uses assessment content, formats, and accommodations 
consistent with the objectives being assessed. It should be guided by the best 
available research on assessment item design and delivery. The framework should 
include student diversity as reflected in gender, geographic location, language 
proficiency, race/ethnicity, socioeconomic status, and disability. The assessment 
should be designed and written to be accessible by the majority of students, min- 
imizing the need for special accommodations for both students with disabilities 
and English language learners. However, students with special needs should be 
provided appropriate accommodations to allow them to participate in the assess- 
ment. The framework should reflect knowledge about the acquisition of key sci- 
ence concepts over time, based on research about how students learn. Critical con- 
tent and skills should be articulated and assessed across grades 4, 8, and 12 (verti- 
cally), as well as across the fields of science (horizontally) by creating items that 
are deliberately layered to achieve these goals. 


e The framework uses a variety of assessment formats. This includes well- 
constructed multiple-choice and open-ended items as well as performance tasks. 
In addition, multiple methods of assessment delivery should be considered, 
including appropriate uses of computer technology. 


e Each achievement level—Basic, Proficient, and Advanced—includes a range 
of items assessing various levels of cognitive knowledge that is broad enough 
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to ensure each knowledge level is measured with the same degree of 
accuracy. Descriptions of Basic, Proficient, and Advanced must be as clear as 
possible. 


e Connections among the framework, the specifications, and the assessment 
items themselves are transparent, coherent, and have a consistent level of 
specificity. The specifications should be written with detail consistent with the 
framework. The content addressed in the specifications should reflect the stand- 
ards and focus on the significant information and knowledge that students should 
retain over time (e.g., big ideas, fundamental understandings). The verbs used in 
the specifications should describe the expected target for assessment (e.g., 
identify, describe, evaluate, relate, analyze, and demonstrate). The content expec- 
tations should match in level of specificity and scope across the disciplines. The 
specifications should follow the idea of learning progressions. To assess overarch- 
ing concepts or themes, the specifications should reflect a layered understanding 
of growth in knowledge of the concepts. 


e The framework addresses the use of assessment data to conduct research on 
science learning and to improve science achievement. Data (background varia- 
bles) from the assessment should be collected in a way that provides details about 
the characteristics of the students being assessed (e.g., race/ethnicity and gender), 
the academic preparation of their teachers, and the nature of their schools. Such 
data provide feedback to educators for improving science instruction and learning. 


USES OF NAEP DATA 


For more than four decades, NAEP has provided information integral to reporting on the 
condition and progress of education at grades 4, 8, and 12 for the nation and, more 
recently, for the states and for a set of large, urban school districts. Legislation concern- 
ing NAEP states that its purpose is to provide, in a timely manner, a fair and accurate 
measurement of student academic achievement and reporting of trends in such achieve- 
ment in reading, mathematics, and other subject matter (Public Law 107-279). 


Because of its rigorous design and methodology, NAEP reports are increasingly used for 
monitoring the state of education in the subjects that are assessed, as models for design- 
ing other large-scale assessments, and for secondary research purposes. 


MONITORING 


As the nation’s only ongoing survey of students’ educational progress, NAEP has 
become an increasingly important resource for obtaining information on what students 
know and can do. Because the information it generates is available to policymakers, 
educators, and the public, NAEP can be used as a tool for monitoring student 
achievement in reading, mathematics, science, and other subjects at the national, state, 
and selected district levels. For example, NAEP reports (known as The Nation’s Report 
Card) compare student performance in a given subject across states, within the subject 
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over time, or among groups of students within the same grade. NAEP also reports long- 
term achievement trends for 9-, 13-, and 17-year-olds in reading and mathematics (e.g., 
Perie, Moran, and Lutkus 2005). To the extent that individual state standards reflect the 
common core of knowledge and skills specified in the framework, state comparisons can 
legitimately be made. If a state has unique standards, any comparison is limited by the 
degree of mismatch between NAEP content and state content. Even with this caveat, 
NAEP still stands as a key indicator of what students know and can do in science at 
grades 4, 8, and 12. 


MODEL OF ASSESSMENT DEVELOPMENT AND METHODS 


NAEP assessment frameworks and specifications documents are used as resources for 
international, state, and local curriculums and assessments. The broad-based process used 
in the development of the frameworks and specifications means that current thinking and 
research are reflected in the descriptions of what students should know and be able to do 
in a given subject. In addition, NAEP uses a rigorous and carefully designed process in 
developing the assessment instruments. Pilot tests and internal and external reviews 
ensure that NAEP assessments are reliable and valid with respect to what they attempt to 
accomplish. This sophisticated methodology serves as a model for other assessment 
developers. Given the No Child Left Behind requirement to assess students in science, 
states may wish to use NAEP as one model to guide their own assessment development. 


RESEARCH AND POLICY 


NAEP data include subject-matter achievement results (reported as both scale scores and 
achievement levels) for various subgroups; background information about schools, teach- 
ers, and students at the subgroup level (e.g., course-taking patterns of Hispanic male 12th 
graders); state-level results; reports for a set of large urban districts; history of state and 
district participation; and publicly released assessment questions, student responses, and 
scoring guides. The NAEP website (http://nces.ed.gov/nationsreportcard) contains user- 
friendly data analysis software to enable policymakers, researchers, and others to exam- 
ine all aspects of NAEP data, perform significance tests, and create customized graphic 
displays of NAEP results. These data and software tools can be used to inform policy- 
making and for secondary analyses and other research purposes. 


CHALLENGES OF DEVELOPING A NAEP ASSESSMENT 


There are three major challenges in developing a NAEP framework: (1) measurement 
constraints and the nature of the items included on the assessment, (2) time and resource 
constraints and how much can be assessed in NAEP, and (3) the timeline for the 
framework and the difficulty of developing a 10-year framework with the rapid explosion 
of knowledge in the Information Age. Each of these challenges is discussed below. 
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MEASUREMENT CONSTRAINTS 


Like any large-scale assessment in education, the workplace, or clinical practice, NAEP 
is constrained in what it can measure. This has implications for the proper interpretation 
of NAEP Science Assessment results. The NAEP Science Framework is an assessment 
framework, not a curriculum framework. Although the two are clearly interrelated, each 
has a different purpose and a different set of underlying assumptions. A curriculum 
framework is designed to inform instruction, to guide what is taught, and often, to guide 
how it is taught. It represents a very wide universe of learning outcomes from which 
teachers pick and choose what and how they teach. An assessment framework is a subset 
of the achievement universe from which assessment developers must choose to develop 
sets of items that can be assessed within time and resource constraints. Hence, the science 
content to be assessed by NAEP has been identified as that considered central to the 
Physical, Life, and Earth and Space Sciences. As a result, some important outcomes of 
science education that are difficult and time consuming to measure (such as habits of 
mind, sustained inquiry, and collaborative research), but valued by scientists, science ed- 
ucators, and the business community, will be only partially represented in the framework 
and in the NAEP Science Assessment. Moreover, the wide range of science standards in 
the guiding national documents that could be incorporated into the framework had to be 
reduced in number so as to allow some in-depth probing of fundamental science content. 
As a result, the framework and the specifications represent a careful distillation that is not 
a complete representation of the original universe of achievement outcomes desirable for 
science education. 


TIME AND RESOURCE CONSTRAINTS 


What NAEP can assess is limited by time and resources. Like most standardized assess- 
ments, NAEP is an “on-demand” assessment. It ascertains what students know and can 
do in a limited amount of time (50 minutes for paper-and-pencil questions and, for a sub- 
set of students sampled, an additional 30 minutes for hands-on performance or interactive 
computer tasks) and with limited access to resources (e.g., reference materials, feedback 
from peers and teachers, opportunities for reflection and revision). The national and state 
standards, however, contain goals that require extended time (days, weeks, or months). 
Therefore, to assess student achievement in the kinds of extended activities that are a 
central feature of the national and state standards and many science curricula, it would be 
necessary to know (for example) the quality of students’: 


e reasoning while framing their research questions; 
e planning for data collection and the execution of the plan; 


e abilities to meet unpredictable challenges that arise during an actual, ongoing 
scientific investigation; 


e lines of argument in deciding how to alter their experimental approach in light of 
new evidence; 
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e engagement with fellow students and/or the teacher in interpreting an observation 
or result and deciding what to do about it; and 


e deliberations and reasoning when settling on the defensible conclusions that might 
be drawn from their work. 


Like other on-demand assessments, NAEP cannot be used to draw conclusions about 
student achievement with respect to the full range of goals of science education. States, 
districts, schools, and teachers can supplement NAEP and other standardized assessments 
to assess the full range of science education standards. In addition to describing the 
content and format of an examination, assessment frameworks like this one signal to the 
public and to teachers the elements of a subject that are important. The absence of 
extended inquiry in NAEP, however, is not intended to signal its relative importance in 
the curriculum. Indeed, because of the significance of inquiry in science education, the 
framework promotes as much consideration of inquiry as can be accomplished within the 
time and resources available for assessment. 


BALANCING CURRENT AND FUTURE CURRICULA 


The framework attempts to strike a balance between what can reasonably be predicted 
about future school science and what students are likely to encounter in their curriculum 
and instruction now and in the near future. It is a significant challenge to write a frame- 
work for the future. Cutting-edge science research creates new knowledge at the intersec- 
tion of disciplinary boundaries. For example, research on human and natural systems has 
generated new understanding about environmental science that is closely linked to 
knowledge generated in the Physical, Life, and Earth and Space Sciences. Although the 
framework is organized into the more traditional Physical, Life, and Earth and Space 
Sciences, features of current science research are woven throughout. 


Another example of burgeoning knowledge relates to technology and technological 
design and the role of both in the NAEP Science Assessment. Technology and technolog- 
ical design are closely interrelated with science, yet the focus of this assessment is on sci- 
ence. Hence, technology and technological design are included in the framework but are 
limited to that which has a direct bearing on the assessment of students’ science achieve- 
ment (see chapter three). 


The framework is intended to be both forward looking (in terms of the science content 
that will be of central importance in the future) and reflective (in terms of current school 
science). Because it is impossible to predict with certainty the shape of school science, 
the choices made for this framework should be revisited in response to future develop- 
ments in school science. 
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ACHIEVEMENT LEVELS 


Public Law 107—279 specifies the Governing Board’s responsibilities regarding NAEP, 
including the identification of appropriate achievement goals for each age and grade in 
the subject areas assessed by NAEP. 


To carry out its mandated responsibility to set appropriate achievement goals for NAEP, 
the Governing Board adopted an achievement levels policy in 1989 (modified in 1993). 
This policy establishes three levels of achievement—Basic, Proficient, and Advanced. 
Basic denotes partial mastery of prerequisite knowledge and skills that are fundamental 
for proficient work at each grade. Proficient represents solid academic performance for 
each grade assessed. Students reaching this level have demonstrated competency over 
challenging subject matter, including subject-matter knowledge, application of such 
knowledge to real-world situations, and analytical skills appropriate to the subject matter. 
Advanced signifies superior performance. These levels are the primary means of report- 
ing NAEP results to the general public and policymakers regarding what students should 
know and be able to do on NAEP assessments. (See appendix B for the NAEP science 
achievement level descriptions and additional information about their development and 
use.) 


INTRODUCTION TO THE ASSESSMENT FRAMEWORK 


Science comprises both content and practices. The NAEP Science Assessment provides a 
snapshot view of what the nation’s 4th, 8th, and 12th graders know and can do in science. 
One expects students, as a result of their education and life experiences, to have learned 
about the principles (along with the facts, concepts, laws, and theories) that have been 
verified by the community of scientists, as well as how scientists discover regularities in 
the natural world.’ NAEP will assess students’ abilities to identify and use science 
principles as well as use scientific inquiry and technological design (see chapter three). 
Although the framework distinguishes content from practice, the two are closely linked in 
assessment as in science itself. 


The framework addresses scientific knowledge and processes. Science is a way of know- 
ing about the natural world based on tested explanations supported by accumulated 
empirical evidence. Explanations of natural phenomena that rely on nonscientific views 
are not reflected in the framework. The committees responsible for the framework devel- 
opment relied on National Standards, Benchmarks, international frameworks, and state 
standards for content about the nature and practice of science. As stated in the National 
Standards: 


Scientific explanations must meet certain criteria. First and foremost, they 
must be consistent with experimental and observational evidence about 
nature, and must make accurate predictions, when appropriate, about 


' “Science principles” is used throughout this framework to denote not only the principles but also the facts, 
concepts, theories, and laws of science. 
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systems being studied. They should also be logical, respect the rules of ev- 
idence, be open to criticism, report methods and procedures, and make 
knowledge public (p. 201). 


The design of the NAEP Science Assessment is guided by the framework’s descriptions 
of the science content and practices to be assessed. Exhibit 1 illustrates how content and 
practices are combined (crossed) to generate performance expectations. The columns 
contain the science content (defined by content statements—propositions that express 
science principles—in three broad areas) and the rows contain the four science practices. 
A double dashed line distinguishes identifying science principles and using science prin- 
ciples from using scientific inquiry and using technological design. The first two practic- 
es can be generally considered as “knowing science” and the last two practices can be 
considered as the application of that knowledge to “doing science” and “using science to 
solve real-world problems.” The cells at the intersection of content (columns) and prac- 
tices (rows) contain student performance expectations. Because content and practice cat- 
egories are not entirely distinct, some overlap in the resultant performance expectations is 
to be expected (as denoted by dashed lines). 


Exhibit 1. Crossing content and practices to generate performance expectations 


Science Content 
Physical Science Life Science eee 
Content Statements Content Statements 
Content Statements 
usuityyhye Performance Performance Performance 
Science : : : 
es Expectations Expectations Expectations 
Principles 
g Poe Performance Performance Performance 
u | Science : : : 
= An Expectations Expectations Expectations 
2 | Principles 
to 
a0 
[-7) 
= Usin: 
sy : g : Performance Performance Performance 
9 | Scientific ; ? ; 
n : Expectations Expectations Expectations 
Inquiry 
in 
ang : Performance Performance Performance 
Technological : : ; 
: Expectations Expectations Expectations 
Design 


Exhibit 2 illustrates the fuller process of generating assessment items and interpreting 
student responses. An item is an individual question or exercise on the NAEP Science 
Assessment and is used to gather information about students’ knowledge and abilities. 
Exhibit 2 begins with student performance expectations, which describe in observable 
terms what students are expected to know and do on the assessment. These performance 
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expectations guide the development of assessment items. The cognitive demands (see 
chapter three) of the items can then be used to interpret students’ responses as evidence 
of what students know and can do in science (see Ruiz-Primo et al. 2001 for more infor- 
mation on cognitive interpretations of performances on assessment tasks). Exhibit 2 
suggests a linear process, but the development of an assessment is iterative (e.g., assess- 
ment items are modified based on student responses provided on trials of pilot versions). 


Exhibit 2. Generating items and interpreting responses 


Performance 


Expectations 


generate 


Assessment 
Items 


Items place 
cognitive demands 
on students. These 
cognitive demands 


that elicit 


Student 


Responses 


can be used as tools 

for designing items 

and interpreting that are interpreted 
student responses. as evidence of 


“y What Students 
Know and Can 
Do in Science 


OVERVIEW OF FRAMEWORK CHAPTERS 


This section provides an overview of the content in each chapter of the framework. For 
ease of understanding, three types of textboxes are used throughout the framework: clari- 
fication, crosscutting content, and illustrative item. Clarification textboxes provide details 
on potentially confusing topics, such as the distinction between mass and weight. Cross- 
cutting content textboxes provide descriptions of science content that cuts across the three 
broad content areas, such as biogeochemical cycles. Illustrative item textboxes provide 
assessment items that exemplify recommendations discussed in the text. 
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CHAPTER Two: SCIENCE CONTENT 


Key principles as well as facts, concepts, laws, and theories that describe regularities in 
the natural world are presented in chapter two as a series of content statements to be 
assessed at grades 4, 8, and 12. Taken together, these statements comprise the NAEP 
science content. They define only what is to be assessed by NAEP and are not intended to 
serve as a science curriculum framework. The content statements should be revisited 
periodically in response to new developments in science research and school curriculum. 


The content statements are organized according to the three broad content areas that gen- 
erally comprise the K-12 school curriculum: 


e Physical Science 
e Life Science 
e Earth and Space Sciences 


Classifying content statements into one content area is not always clear-cut, but doing so 
helps ensure the areas of science are assessed in a balanced way. Some common content 
is found to be significant in each of the three content areas (e.g., energy conservation and 
its associated principles are applicable to the living and nonliving systems studied by 
physical, life, and Earth scientists). This crosscutting content is further described in 
chapter two. 


The content statements are derived from National Standards and Benchmarks and are 
informed by international frameworks and state standards. Content statements have been 
added where warranted by advances in science since the development of the standards 
documents. The selection of science content statements to be assessed at each grade level 
focuses on principles central to each discipline, tracks related ideas across grade levels, 
and limits the breadth of science knowledge to be assessed. The selection of content 
statements used an iterative approach and took into account the many perspectives of var- 
ious stakeholders. 


CHAPTER THREE: SCIENCE PRACTICES 


The following science practices are found in most or all of the major sources used to 
develop the framework (see standards documents listed above). The practices to be as- 
sessed at grades 4, 8, and 12 are organized into four categories: 


Identifying Science Principles 
Using Science Principles 
Using Scientific Inquiry 
Using Technological Design 


As shown in exhibit 1, the Physical Science, Life Science, and Earth and Space Sciences 
content statements listed in chapter two can be deliberately combined (crossed) with each 
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of the above four practices to generate specific performance expectations. These perfor- 
mance expectations are written in observable terms and guide the development of 
assessment items. Performance expectations thus provide an outline of what the NAEP 
Science Assessment will expect as evidence of what students know and can do. At the 
end of chapter three, examples of performance expectations, items, and interpretations of 
student responses are provided. (See the specifications for more examples.) 


Students’ understanding increases over time as they learn more, moving from initially 
naive to increasingly more sophisticated knowledge about the natural world. These learn- 
ing progressions are further described in chapter three. 


CHAPTER FOUR: OVERVIEW OF THE ASSESSMENT DESIGN 


Chapter four provides an overview of the assessment design, including a discussion of 
assessment item contexts, types of items, distribution of items, accessibility concerns for 
English language learners and students with disabilities, and recommendations for small- 
scale special studies. 


Beyond the science content statements and practices, there are other valuable components 
of science that NAEP will not directly assess. These components—the history and nature 
of science and the relationship between science and technology—are treated as possible 
contexts in which assessment items may be presented. 


The assessment includes selected-response (multiple-choice) items and constructed- 
response items (which include short and extended constructed-response items as well as 
concept-mapping tasks). Some combination items may require more than one response. 
They include item clusters, predict-observe-explain (POE) item sets, hands-on perfor- 
mance tasks, and interactive computer tasks. The responses requested may be all selected 
response, all constructed response, or a mixture of both. (See chapter four for a fuller 
explanation of types of items, including examples.) 


Chapter four recommends three types of percentages for item distribution (as measured 
by student response time) at each grade level: 


e Items by content area (Physical, Life, and Earth and Space Sciences) 


e Items by science practice (Identifying Science Principles, Using Science Princi- 
ples, Using Scientific Inquiry, and Using Technological Design) 


e Items by type (selected-response and constructed-response items) 


Additional specifications about the number of hands-on performance tasks and interactive 
computer tasks are also provided. 


Chapter four also describes considerations for English language learners and students 
with disabilities. In particular, NAEP assessments need to be responsive to growing de- 
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mands for increased inclusion of all types of students in the general curriculum and in- 
creased emphasis and commitment to serve and be accountable for all students. A number 
of small-scale special studies also are recommended at the end of chapter four. 


COMPARISON OF 1996-2005 NAEP SCIENCE FRAMEWORK AND 2009-2015 
NAEP SCIENCE FRAMEWORK 


The chapter overviews provided above reflect the differences between the NAEP 1996-— 
2005 science framework and the framework to be used for the 2009, 2011, and 2015 
NAEP science assessments. The assessments resulting from this framework start a new 
NAEP science trend (i.e., measure of student progress in science). One difference 
between the two frameworks is that the 2009 steering and planning committees drew on a 
variety of new standards and assessments. They also were able to extract findings from 
research in science, science education, and cognition and to consider the use of technolo- 
gy to increase the options for assessment administration. Exhibit 3 lists the major differ- 
ences between the 1996—2005 and 2009-2015 NAEP science frameworks. 


Exhibit 3. Comparison of NAEP science frameworks: 1996—2005 and 2009-2015 
1996-2005 Framework 2009-2015 Framework 

Content drawn from existing standards and 
Few science standards available on which to | assessment frameworks: National 

base the content to be assessed Standards, Benchmarks, TIMSS, PISA, and 
state standards 

Content areas organized into Physical 
Science, Life Science, and Earth and Space 
Sciences 

Recommendations on distribution of 
questions by science content area and grade: 
equal weight for all three sciences in grade 
4; emphasis on Earth and Space Sciences in 
grade 8; emphasis on Physical Science and 
Life Science at grade 12 

Content presented as statements in tables 
Content presented as bullets and short organized by science content subtopics (e.g., 
phrases forces affecting motion from Physical 
Science) and by grade level 


Content areas organized into Physical 
Science, Life Science, and Earth Science 


Recommendations on distribution of 
questions by fields of science and grade: 
approximately equal distribution in grades 4 
and 12; a somewhat heavier emphasis on 
Life Science in grade 8 
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Framework employed three abstract themes: 
systems, models, and patterns of change 


Framework employs crosscutting content 
among Life, Physical, and Earth and Space 
Sciences 


Assessment asked questions about the nature 
of science 
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Exhibit 3 (continued). Comparison of NAEP science frameworks: 1996-2005 and 
2009-2015 


1996-2005 Framework 


2009-2015 Framework 
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Assessment Items 


Knowing and doing dimension organized 
into conceptual understanding, scientific 
investigation, and practical reasoning 


Science practices dimension organized into 
Identifying Science Principles, Using 
Science Principles, Using Scientific Inquiry, 
and Using Technological Design 


Nature of science treated within science 
practices, particularly Using Science 
Principles and Using Scientific Inquiry 


Science practices assessed were largely 
experience based 


Science practices assessed take into account 
extant research and the cognitive complexity 
of the items 


Assessment included items on practical 
reasoning (i.e., applying science to suggest 
effective solutions to everyday problems) 


Assessment includes questions on 
technological design (i.e., the systematic 
process of applying science knowledge and 
skills to solve problems in a real-world 
context) 


Forty-five percent of the assessment focused 
on conceptual understanding 


Sixty percent of the assessment focuses on 
conceptual understanding (Identifying and 
Using Science Principles) 


Assessment included both paper-and-pencil 
and hands-on performance tasks 


Learning progressions (i.e., connected 
sequences of science performances across 
grade spans) are included 


Assessment uses the history of science and 
the relationship between science and 
technology as contexts for questions 


Assessment includes paper-and-pencil 
questions, hands-on performance tasks, and 
interactive computer tasks 


No illustrative items to convey science 
knowledge or practices in the framework; 
only a few suggested ideas for items 
provided in the specifications 


Illustrative items that convey science 
knowledge and practices are included in 
both the framework and the specifications 


Framework and specifications include 
guidelines for assessing students with 
disabilities and English language learners 


Framework includes examples showing how 
questions are generated and interpreted 


Students’ naive conceptions about science 
principles are explicitly assessed 
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CHAPTER Two 
SCIENCE CONTENT 


This chapter presents a series of statements that describe the science content of the NAEP 
Science Assessment. The content statements contain key science principles for NAEP 
assessment. In the framework, the phrase “science principles” is broadly conceived and 
encompasses not only the key principles but also the facts, concepts, laws, and theories of 
science. To specify the science that should be assessed at each grade level, the framework 
organizes the science content into the three broad content areas that generally make up 
the K-12 school curriculum to which students are exposed: 


e Physical Science 
e Life Science 
e Earth and Space Sciences 


Classifying statements into one primary content area is not always clear-cut and is artifi- 
cial to some extent. For example, physicist Emest Rutherford’s discovery of the nucleus 
earned him the Nobel Prize in Chemistry and physicist Rosalyn Yalow’s work on radio- 
immunoassay earned her the Nobel Prize in Medicine. However, using three broad 
content areas as an organizer helps ensure that key science content is assessed in a bal- 
anced way. 


For clarity, exhibits are used to depict the content statements at each grade level. The 
content statements are based on the assumption that a person literate in science is one 
who understands key science ideas, is aware that science and technology are interdepend- 
ent human enterprises with strengths and limitations, is familiar with the natural world 
and recognizes both its diversity and unity, and uses scientific knowledge and ways of 
thinking for individual and social purposes (see AAAS 1994, p. xvii). 


Two types of textboxes are used throughout this chapter. Clarification textboxes provide 
details on potentially confusing content, such as the distinction between mass and weight. 
Crosscutting content textboxes provide descriptions of science content that cuts across 
the three broad content areas, such as biogeochemical cycles. 


DEVELOPMENT OF THE CONTENT STATEMENTS 


The selection and generation of specific content statements at each grade level followed a 
similar approach across the three broad content areas: 


e The National Standards and Benchmarks were used as key documents for 
identifying the science content to be assessed, pursuant to the charge from the 
steering committee. Various tools, primarily crosswalks between the National 
Standards and Benchmarks (AAAS 1997; Kendall and Marzano 2004), were used 
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to cross-check the documents’ content standards and benchmark statements; those 
that were common to both documents were generally given priority. On a case- 
by-case basis, content not represented in both documents was discussed and 
decisions made about inclusion or exclusion. Additions were made where 
warranted by scientific advances in the decade or more since the development of 
the National Standards and Benchmarks, or as a consequence of international 
assessment results from TIMSS and PISA. Some of this framework’s content 
statements are verbatim reproductions of statements from the National Standards 
and Benchmarks.” 


e The focus in the selection process was on the central principles of each discipline. 
The content statements in the framework represent foundational and pervasive 
knowledge, key points of scientific theories, and underpinnings upon which com- 
plex understanding is built; and/or they demonstrate connectivity to other central 
content. 


e A primary consideration was the grade-level appropriateness and accuracy within 
grade level of content statements. 


e Once key content was identified within subtopics, the progression of ideas and 
performances, informed by available research, was tracked through grades 4, 8, 
and 12. 


e A deliberate attempt was made to limit the breadth of science content to be 
assessed so that some important topics could be measured in-depth. Once core 
content was identified in each science area, additional content statements could be 
added only if others previously included were eliminated. 


The selection and generation of content statements for inclusion in the framework was 
not a linear process. Although the planning committee attempted to use clear and concise 
language, the complexities associated with the task of defining what students should 
know and be able to do in science at particular points in their development necessitated 
an iterative approach that included many perspectives. In addition to internal review by 
the project committees and staff, outreach activities gathered external feedback on the 
content statements from a variety of stakeholders (e.g., teachers, school and district ad- 
ministrators, state science education personnel, policymakers, scientists, and members of 
the business, industry, and postsecondary communities). The framework should be revis- 
ited in the future as new research becomes available and as the influence of new devel- 
opments in science takes shape in the K-12 curriculum. 


* Statements are reprinted from National Science Education Standards (NRC 1996) with permission from 
the National Academy of Sciences, courtesy of the National Academies Press, Washington, D.C. State- 
ments are reprinted from Benchmarks for Science Literacy (AAAS 1993) with permission from Project 
2061, on behalf of the American Association for the Advancement of Science, Washington, D.C. 
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ORGANIZATION OF SCIENCE CONTENT 


As described above, this framework organizes science content into three broad content 
areas (Physical Science, Life Science, and Earth and Space Sciences). The content is 
further organized into topics (such as motion), subtopics (such as forces affecting 
motion), and grade-specific content statements. The description of each broad content 
area follows this structure of increasing specificity and is presented in two ways: narra- 
tive introductions and content statements presented in exhibits (see exhibits 8, 10, and 12 
in the sections for Physical Science, Life Science, and Earth and Space Sciences, respec- 
tively). Exhibit 4 summarizes the NAEP science content topics and subtopics. 


Exhibit 4. NAEP science content topics and subtopics 


Physical Life Earth and Space 
Science Science Sciences 
Matter Structures and Functions | Earth in Space and Time 
e Properties of matter | of Living Systems e Objects in the 
e Changes in matter e Organization and universe 
development e History of Earth 
Energy e Matter and energy 
e Forms of energy transformations Earth Structures 
e Energy transfer and e Interdependence e Properties of Earth 
conservation materials 
Changes in Living e Tectonics 
Motion Systems 
e Motion at the e Heredity and Earth Systems 
macroscopic level reproduction e Energy in Earth 
e Forces affecting e Evolution and systems 
motion diversity e Climate and weather 
e Biogeochemical 
cycles 


As an organizational tool in exhibits 8, 10, and 12, each content statement is preceded by 
a specific code in bold (e.g., “12.10: Sorting and recombination of genes in sexual 
reproduction results in a great variety of possible gene combinations from the offspring 
of any two parents”). Within a code, the letter denotes broad content area (“P” for Physi- 
cal Science, “L” for Life Science, and “E” for Earth and Space Sciences); the number 
before the period denotes grade level (grade 4, 8, or 12); and the number following the 
period denotes the content statement’s order of appearance within a given content area 
and grade. Thus, L12.10 denotes that this is the tenth content statement to appear in the 
grade 12 section of the Life Science content statements table. Because the numbering 
within each content area and grade is strictly sequential, code numbers do not necessarily 
indicate any relationships across grades (see, for example, P4.13, P8.13, and P12.13). 
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INTERPRETATION OF THE CONTENT STATEMENTS 


In the framework, the content statements generally follow a form that is consistent with 
the National Standards, Benchmarks, and the practice of the scientific community. The 
content statements are phrased as propositions that express science principles. Based on 
evidence, these principles have been verified by the scientific community and are under 
constant review. Exhibit 5 is an example of how one grade 8 Physical Science principle is 
represented in the framework, National Standards, and Benchmarks. 


Exhibit 5. One grade 8 Physical Science principle represented in the framework, 
National Standards, and Benchmarks 


Framework National Standards Benchmarks 
P8.16: Forces have magnitude If more than one force acts onan | An unbalanced force acting on an 
and direction. Forces can be object along a straight line, then object changes its speed or 
added. The net force on an object | the forces will reinforce or cancel | direction of motion, or both. If 
is the sum of all the forces acting | one another, depending on their the force acts toward a single 
on the object. A nonzero net directions and magnitude. center, the object’s path may 
force on an object changes the Unbalanced forces will cause curve into an orbit around the 
object’s motion; that is, the changes in the speed or direction | center (p. 90). 


object’s speed and/or direction of | of an object’s motion (p. 154). 
motion changes. A net force of 
zero on an object does not change 
the object’s motion; that is, the 
object remains at rest or 
continues to move at a constant 
speed in a straight line (p. 35).* 


* This content statement is longer in the framework than in the Benchmarks or National Standards—not 
because it introduces additional science principles, but because it has adopted more detailed language. 


The content statements form the basis for explaining or predicting naturally occurring 
phenomena. For example, the above content statement about objects in motion can be 
used to explain and predict the motions of many different specific objects (e.g., an ice 
skater, an automobile, an electron, or a planet). 


The content statements do not include observations of phenomena. As the content state- 
ments are written, the empirical foundations of the science principles they represent are 
not detailed. Instead, knowledge is presented in general terms, such as patterns in obser- 
vations or theoretical models that account for these patterns. The NAEP assessment will 
require students to apply content statements to specific observations and examples of 
phenomena. Therefore, the specifications provide boundaries on the range of phenomena 
and examples that are appropriate for inclusion on the NAEP Science Assessment. The 
boundaries also describe the appropriate instrumentation, representations, and technical 
vocabulary to be included in the assessment. 


The process of item development and making inferences about students’ knowledge and 
abilities (see exhibits 1 and 2) may necessitate further clarification of the content state- 
ments themselves. For example, this may involve “detailing” the meanings of individual 
content statements (e.g., “boiling point” assumes standard atmospheric pressure) or 
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further defining the boundaries of the content to be assessed (e.g., grade 12 students are 
expected to know that DNA provides instructions for assembling proteins but not the 
details of DNA transcription and translation). Exhibit 6 is an example of commentary on 
a grade 8 Physical Science content statement. Although the commentary describes key 
ideas about waves relevant through grade 12, only three of these key ideas are recom- 
mended for assessment at grade 8. Whenever possible, commentary on content state- 
ments was informed by available research on learning and cognition. Commentary is 
intended as helpful notes to item writers and is provided for a number of content state- 
ments in the Specifications. Commentary is not always of the same kind, and the 
examples provided do not exhaustively cover the content statements. Assessment devel- 
opers should continue the process of “detailing” the grade-appropriate principles to be 
assessed for all content statements sampled for a particular NAEP Science Assessment. 


Exhibit 6. Commentary on a Physical Science content statement 


Grade 8: Energy—Forms of Energy 


Content Statement 


P8.10: Energy is transferred from place to place. Light energy from the Sun travels 
through space to Earth (radiation). Thermal energy travels from a flame through the metal 
of a cooking pan to the water in the pan (conduction). Air warmed by a fireplace moves 
around a room (convection). Waves—including sound and seismic waves, waves on 
water, and light waves—have energy and transfer energy when they interact with matter. 


Commentary 
Wave principles recommended for assessment at grade 8: 


e Waves involve transfer of energy without a transfer of matter. 

e Waves are caused by disturbances and are also themselves disturbances. Some of 
the energy of these disturbances is transmitted by the wave. 

e Water, sound, and seismic waves transfer energy through a material. 
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Exhibit 6 (continued). Commentary on a Physical Science content statement 
Wave principles that are related but not recommended for assessment at grade 8: 


Some waves are transverse (water, seismic) and other waves are longitudinal 
(sound, seismic). 

In transverse waves, the direction of the motion is perpendicular to the 
disturbance. 

In longitudinal waves, the direction of motion is parallel to the disturbance. 
Waves (e.g., light waves) traveling from one material to another undergo 
transmission, reflection, and/or changes in speed. 

Waves can be described by their wavelength, amplitude, frequency, and speed 
(speed is frequency multiplied by wavelength; energy is a function of the 
amplitude for nonelectromagnetic waves). 

Light has dual wave-particle properties. 


Energy and refraction calculations are not recommended for assessment at grade 8. A 
quantitative understanding of electromagnetic waves is expected at grade 12. See P12.10. 


To fully understand the content statements and their intent, readers of the framework 
should be cognizant of the following: 


e Although all content statements have been assigned a primary classification, some 
are likely to fall into more than one content area. 


e Some assessment items may draw on more than one content statement at a time. 


e Empty cells in the content statement tables denote that a particular subtopic is not 
recommended for assessment at that grade level. 


e Retention of foundational knowledge from one grade to the next is assumed; 
however, if the relevant content statement does not appear in a succeeding grade 
level, it should not be assessed. 


e The content statements listed in the framework describe the whole of what is to be 
assessed on the NAEP Science Assessment. The content statements should not be 
interpreted as a complete description of the recommended school science curricu- 
lum. 


CROSSCUTTING CONTENT 


Scientists define their specializations narrowly (e.g., astronomy, molecular biology, 
organic chemistry) to organize their research communities; the categories of Physical 
Science, Life Science, and Earth and Space Sciences are helpful for organizing school 
science. These categorizations mask the fact that some science principles cut across the 
content areas. In this framework, crosscutting content is not represented by abstractions 
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such as “models,” “constancy and change,” or “form and function,” but is anchored in the 
content statements themselves. Some examples of crosscutting content are described in 
textboxes that appear throughout the content area introductions: Energy Sources and 
Transfer in Physical Science; Uses, Transformations, and Conservation of Energy in Life 
Science; and Biogeochemical Cycles in Earth and Space Sciences. Additional examples 
are suggested in the specifications. For instance, the theory of plate tectonics and the 
evolution of Earth’s surface are inextricably linked with environmental pressures (such as 
geographic barriers), speciation, and the evolution of life. Such examples illustrate 
opportunities for assessing specific content in greater depth. 


PHYSICAL SCIENCE 


Physical Science principles, including fundamental ideas about matter, energy, and 
motion are powerful conceptual tools for making sense of phenomena in physical, living, 
Earth, and space systems. Familiar changes—an ice cube melting, a baseball changing 
direction after being struck by a bat, the appearance of a bolt of lightning, the formation 
and erosion of mountains, and the growth of a plant—can be explained using these 
fundamental ideas. ° 


Energy is the constant in an ever-changing world. Energy from the Sun fuels electrical 
storms, hurricanes, tornados, and photosynthesis. In turn, the products of photosynthesis 
(carbohydrates and oxygen) react during respiration to fuel life processes, such as growth 
and reproduction of plants and animals. Consequently, it is important for students to de- 
velop an understanding of Physical Science principles early and to appreciate their use- 
fulness across Physical Science, Life Science, and Earth and Space Sciences. 


The Physical Science principles to be assessed are divided into three topics—matter, 
energy, and motion. Matter is the “stuff” of the natural world. Energy is involved in all 
changes in matter. Motion of the heavenly bodies, of objects found in daily experiences 
(e.g., balls, birds, cars), and of the tiny particles (atoms, molecules, and their component 
parts) composing all objects and substances is the result of interactions of matter and en- 
ergy. The content statements have been divided into topics and subtopics as summarized 
in exhibit 7. 


° The importance of developing an understanding early and making connections among the Physical, Life, 
and Earth and Space Sciences has led to increased attention on Physical Science in elementary school and 
prompted consideration of rearranging the usual Earth—life—chemistry—physics curriculum sequence. 
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Exhibit 7. Physical Science content topics and subtopics 
Matter 


Properties of matter 
Changes in matter 


Energy 


Forms of energy 
Energy transfer and conservation 


Motion 


Motion at the macroscopic level 
Forces affecting motion 


MATTER 


The topic “matter” is divided into two subtopics: properties of matter and changes in mat- 
ter. Conservation of mass, the particulate model of matter, and the periodic table of the 
elements are the concepts that connect these two subtopics and their related principles. 


Properties of Matter 


Matter has physical and chemical properties. Physical properties common to all matter as 
well as those physical properties unique to solids, liquids, and gases are included in the 
framework, as are chemical properties. All objects and substances in the natural world are 
composed of matter. Matter has two fundamental properties: it takes up space and it has 
inertia; that is, it changes motion only when under the influence of a nonzero net force 
(grade 4). See the following textbox, “Clarification: A Matter of Mass,” for more on mass 
versus weight and the framework’s treatment of this distinction. 
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Clarification: A Matter of Mass 


Mass is a property common to all objects. It is the amount of matter (or “stuff”) in an 
object. The more mass an object has, the more inertia (or sluggishness) it displays when 
attempts are made to change its speed or direction. Mass is measured in grams (g) or 
kilograms (kg) (1 kg = 1,000 g) using a beam or electronic balance.* 


Weight, on the other hand, is a measure of the force of attraction (gravitational force) 
between an object and Earth. Every object exerts gravitational force on every other 
object. The force depends on how much mass the objects have and on how far apart they 
are. Force and weight are measured in newtons (N) using a spring scale. 


Changing an object’s position (e.g., from Earth to the Moon) will change its weight, but 
not its mass. For example, on the surface of Earth, a cannonball has a mass of 10 kg anda 


weight of 98 N. On the surface of the Moon, the cannonball still has a mass of 10 kg, but 
its weight is only 16 N. So, the cannonball weighs less on the Moon than on Earth, even 
though nothing has been taken away. Why? Because of the Moon’s lesser mass and 
smaller radius, the force of attraction between the Moon and the cannon ball is less than 
the force of attraction between Earth and the cannonball. Hence, it is said that an object 
on the Moon weighs less than the same object weighs on Earth. 


These concepts of mass and weight are complicated and potentially confusing to 4th 
grade students. Therefore, this framework uses the more familiar term “weight” in grade 
4 to stand for both weight and mass, and this usage is denoted as “weight (mass).” By 
grades 8 and 12, students are expected to understand the distinction between mass and 
weight, and both terms will be used as appropriate. 


* In current NAEP practice, metric units of measure are used for grades 4, 8, and 12. 


Matter exists in several different physical states, each of which has unique properties 
(grade 4). Three of the most commonly encountered states are solids, liquids, and gases. 
Shape and compressibility are examples of properties that distinguish solids, liquids, and 
gases (grade 4). 


The particulate model of matter can be used to explain and predict the properties of states 
of matter, such as why ice is harder than liquid water and why ice (once formed) has a 
shape independent of its container while liquid water takes the shape of whatever 
container it is in (grade 4). In the particulate model of matter, the molecules or atoms of 
which matter is composed are assumed to be tiny particles in motion (grade 8). The 
motion is translational, rotational, and vibrational (grade 12). This model can be used to 
explain the properties of solids, liquids, and gases as well as changes of state. The 
particulate model can be used to explain the unique properties of water, as described in 
the following textbox. 
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Clarification: Unique Properties of Water 


Grade 12: Matter—Changes in Matter 

P12.5: Changes of state require a transfer of energy. Water has a very high specific heat, 
meaning it can absorb a large amount of energy while producing only small changes in 
temperature. 


The unique properties of water have important consequences for Earth systems and Life 
Science, including the origin and existence of life on Earth. Understanding the substance 
of water requires knowledge across the Physical Science categories of Matter, Energy, 
and Motion. 


As with all kinds of matter, water’s unique properties can be explained by the shape of its 
molecule, the forces between its molecules in solid (ice) and liquid states, and the 
resulting arrangement of molecules in solid and liquid states. In particular, in the solid 
state, the molecular bonds are such that they separate the molecules more than in the 
liquid state, resulting in ice being less dense than liquid water. The strong intermolecular 
forces of liquid water account for its high specific heat. (The specific heat of a substance 
is the amount of energy required to change 1 g of the substance by 1 degree centigrade.) 


The detailed structures of molecules and the atoms that compose them serve as models 
that explain the forces of attraction between molecules. The structure of atoms, especially 
the outermost electrons, explains the chemical properties of the elements and the for- 
mation of the chemical bonds made and broken during chemical reactions (grade 12). The 
Periodic Table of the Elements (introduced at grade 8) is another way in which order can 
be made out of the complexity of the variety of types of matter. (In grade 8, the emphasis 
is on observed periodicity of properties.) The Periodic Table demonstrates the relation- 
ship between the atomic number of the elements and their chemical and physical proper- 
ties and provides a structure for inquiry into the characteristics of the chemical elements 
(grade 12). 


Two classes of chemical substances serve as exemplars of chemical properties: metals 
(elements) and acids (compounds). A chemical property of metals is to react with non- 
metals to form salts. One of the properties of acids is the formation of characteristic col- 
ors when interacting with acid/base indicators and the interaction with bases to produce 
salts and water (i.e., neutralization) (grade 8). 


Changes in Matter 


Matter can undergo a variety of changes. Changes are physical if the relationships 
between the molecules of the material are changed, such as from a solid to a liquid or 
from a liquid to a gas (grade 4). When matter undergoes a physical change, generally no 
changes occur in the structure of the molecules or atoms composing the matter (grade 8), 
although there are exceptions (e.g., sulfur). Changes are chemical if they involve the 
rearrangement of how atoms are bound to one another, thereby changing the molecules of 
the material. These are changes in the configuration of the outermost electron shell 
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surrounding the nuclei of the interacting atoms (grade 12). Changes are nuclear if the 
particles are emitted from or absorbed into the nucleus of the atom, thereby changing the 
atoms themselves into isotopes or different elements (grade 12). 


The fact that mass is conserved when matter undergoes physical and chemical changes is 
a powerful principle for understanding the natural world and was influential in the devel- 
opment of chemical theory. Adherence to the principle discourages the conclusion that 
something “disappears” (as water seems to disappear from a puddle) and encourages the 
search for the “missing” matter. 


Most nuclear reactions, involving changes in the nuclei of atoms, are very high energy 
and result in the formation of elements or nuclei different from those that began the 
process. In nuclear reactions, a measurable amount of mass is converted into energy 
(grade 12). 


ENERGY 


The topic “energy” is divided into two subtopics; one addresses the forms of energy and 
the other addresses energy transfer and conservation.* 


Forms of Energy 


Knowing the characteristics of familiar forms of energy (grade 4) and the scientific cate- 
gories of potential and kinetic energy (grade 8) can lead to an understanding that, for the 
most part, the natural world can be explained and is predictable. The most basic charac- 
teristics of thermal, light, sound, electrical, and mechanical energy and the relationship 
between changes in the natural world and energy are included in the framework.” For 
example, the fact that two objects, one at a higher temperature than the other, come to the 
same temperature when placed in contact with each other is a familiar experience. Heat 
as a concept can be used to explain this experience (grade 8). 


Energy Transfer and Conservation 


The fact that energy is conserved can be demonstrated by keeping track of the familiar 
forms of energy as they are transferred from one object to another. The chemical poten- 
tial energy in a battery is transferred by electric current to a light bulb, which in turn 
transfers the energy in the form of heat (thermal energy) and light to its surroundings 
(grade 4). The energy stored in the battery decreases as its surroundings are heated. The 
loss in chemical potential energy equals the light and heat (thermal energy) transferred by 
the bulb and the wires to their surroundings. Quantitative accounting is complex; howev- 
er, on a qualitative basis, both the ability to trace energy transfer and the understanding 


“There are different approaches to helping students understand the concept of energy and related observa- 
ble phenomena, such as light, heat, and sound. These differences are reflected in the framework’s reference 
documents and have therefore influenced the framing of the Physical Science, Life Science, and Earth and 
Space Sciences content statements. 

° The term “heat” in Physical Science is used to stand for thermal energy in grade 4; this usage is denoted 
as “heat (thermal energy).” “Thermal energy” is used in grades 8 and 12. 
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that energy is conserved (grade 8) are of great explanatory and predictive value. Chemi- 
cal reactions either release energy to the surroundings or cause energy to flow from the 
surroundings into the system (grade 12). The Sun as the main energy source for the Earth 
provides an opportunity at all grade levels to make important connections between the 
science disciplines (see the following textbox). 


Crosscutting Content: Energy Sources and Transfer 


When a sufficiently high temperature occurs in the Sun because of gravitational 
attraction, nuclear reactions take place. These reactions result in the transfer of energy 
from nuclei to their surroundings. At the same time, those high temperatures cause the 
Sun to radiate visible light and many other forms of electromagnetic waves. A small 
fraction of this light energy reaches Earth, heating the land, air, and water. Some of this 
energy causes some water to evaporate. The water vapor travels high into the atmosphere, 
thus increasing its gravitational potential energy. There it cools and condenses, some of it 
falling into reservoirs behind dams. At many dams, some of this water is directed 
downhill through tubes, resulting in the transfer of gravitational potential energy to the 
descending water as kinetic energy. This water is then used to turn turbines, and energy is 
transferred from the moving water to electrical appliances through circuits and power 
lines. Accordingly, the energy used to power commonplace objects such as a light bulb, 
TV, radio, or stereo can be traced back to nuclear reactions deep inside the Sun. 


Consider also the transfer of energy that occurs as a diver falls through air into water. 
When the diver is initially poised on a cliff high above a lake’s surface, one says that the 
diver has potential energy with respect to the air and water below. As the diver falls, her 
speed (kinetic energy) increases as her potential energy decreases. Her body transfers 
energy to the medium through which she falls; that is, the diver’s body rubs against the 
air and water (heating them) and exerts force on the air and water (moving them aside). 
When the diver finally comes to rest in the lake, some or most of her potential energy has 
been transferred to heating and setting into motion the air and water through which she 
fell. 


The following grade 12 content statements illustrate the crosscutting nature of energy 
sources and transfer. They are not intended to represent an exhaustive catalog of all 
statements related to this crosscutting content. 


Physical Science Earth and Space Sciences 


P12.11: Fission and fusion are reactions involving | E12.9: Earth systems have internal and external 
changes in the nuclei of atoms. Fission is the sources of energy, both of which create heat. The 
splitting of a large nucleus into smaller nuclei and | Sun is the major external source of energy. Two 
particles. Fusion involves joining of two relatively | primary sources of internal energy are the decay of 
light nuclei at extremely high temperature and radioactive isotopes and the gravitational energy 
pressure. Fusion is the process responsible for the | from Earth’s original formation. 

energy of the Sun and other stars. (Underlining is 
used here and on pp. 42 and 56 to link segments of 


the content statements across content areas.) 
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MOTION 


The topic “motion” is divided into two subtopics; one addresses motion at the macro- 
scopic level and the other addresses the forces that affect motion. 


Motion at the Macroscopic Level 


Objects observed in daily life undergo different kinds of motion (grade 4). The frame- 
work distinguishes three kinds of motion (translational, rotational, and vibrational) and 
emphasizes the translational motion of objects in the natural environment (grade 12). 
Translational motion is more difficult to describe than it appears because descriptions 
depend on the position of the observer and the frame of reference used. Speed (grades 4 
and 8), velocity (grade 12), and acceleration (grade 12) of objects in translational motion 
are described in terms of change in direction and position in a time interval. 


Forces Affecting Motion 


It takes energy to change the motion of objects. The energy change is understood in terms 
of forces. For example, it takes energy for a baseball pitcher to set the ball in motion 
toward the batter. Also, pushes and pulls applied to objects often result in changes in 
motion (grade 4). Principles germane to the relationship of forces and motion serve to 
motivate the search for forces when objects change their motion or when an object 
remains at rest even though it seems that the forces acting on it should result in setting it 
in motion (grade 8). 


Some forces act through physical contact of objects while others act at a distance. The 
force of a bat on a ball and the downward push of a lead block resting on a tabletop are 
contact forces. Gravitational and magnetic forces act at a distance (grade 8). Magnets do 
not need to be in contact to attract or repel each other. The Earth and an airplane do not 
need to be in contact for a force of attraction to exist between them. Qualitative relation- 
ships (grade 8) and quantitative relationships (grade 12) between the mass of an object, 
the magnitude and direction of the net force on the object, and its acceleration are power- 
ful ideas to explain and predict changes in the natural world. 
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Exhibit 8. Ph 


Grade 4 


sical Science content statements for 


rades 4, 8, and 12 


Grade 12 


P4.1: Objects and substances have 
properties. Weight (mass) and volume 
are properties that can be measured 
using appropriate tools.* 


P4.2: Objects vary in the extent to 
which they absorb and reflect light 
and conduct heat (thermal energy) and 
electricity. 


P4.3: Matter exists in several different 
states; the most common states are 
solid, liquid, and gas. Each state of 
matter has unique properties. For 
instance, gases are easily compressed 
while solids and liquids are not. The 
shape of a solid is independent of its 
container; liquids and gases take the 
shape of their containers. 


P4.4: Some objects are composed of a 
single substance; others are composed 
of more than one substance. 


P4.5: Magnets can repel or attract 
other magnets. They can also attract 
certain nonmagnetic objects at a 
distance. 


P8.1: Properties of solids, liquids, and 
gases are explained by a model of 
matter that is composed of tiny 
particles in motion. 


P8.2: Chemical properties of 
substances are explained by the 
arrangement of atoms and molecules. 


P8.3: All substances are composed of 
1 or more of approximately 100 
elements. The periodic table organizes 
the elements into families of elements 
with similar properties. 


P8.4: Elements are a class of 
substances composed of a single kind 
of atom. Compounds are composed of 
two or more different elements. Each 
element and compound has physical 
and chemical properties, such as 
boiling point, density, color, and 
conductivity, which are independent 
of the amount of the sample.’ 


P8.5: Substances are classified 
according to their physical and 
chemical properties. Metals and acids 
are examples of such classes. Metals 
are a class of elements that exhibit 
common physical properties such as 
conductivity and common chemical 
properties such as reacting with 
nonmetals to produce salts. Acids are 
a class of compounds that exhibit 
common chemical properties, 
including a sour taste, characteristic 
color changes with litmus and other 
acid/base indicators, and the tendency 
to react with bases to produce a salt 
and water. 


P12.1: Differences in the physical 
properties of solids, liquids, and gases 
are explained by the ways in which 
the atoms, ions, or molecules of the 
substances are arranged and the 
strength of the forces of attraction 
between the atoms, ions, or 
molecules. 


P12.2: Electrons, protons, and 
neutrons are parts of the atom and 
have measurable properties, including 
mass and, in the case of protons and 
electrons, charge. The nuclei of atoms 
are composed of protons and 
neutrons. A kind of force that is only 
evident at nuclear distances holds the 
particles of the nucleus together 
against the electrical repulsion 
between the protons. 


P12.3: In the Periodic Table, elements 
are arranged according to the number 
of protons (called the atomic number). 
This organization illustrates 
commonality and patterns of physical 
and chemical properties among the 
elements. 


P12.4: In a neutral atom, the 
positively charged nucleus is 
surrounded by the same number of 
negatively charged electrons. Atoms 
of an element whose nuclei have 
different numbers of neutrons are 
called isotopes. 


* See textbox on p. 27 for more information on the distinction between weight and mass. 
‘ Although this content statement generally holds true, some compounds decompose before boiling. 
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Grade 4 


Exhibit 8 (continued). Ph 


sical Science content statements for 


rades 4, 8, and 12 
Grade 12 


P4.6: One way to change matter from 
one state to another and back again is 
by heating and cooling. 


P8.6: Changes of state are explained 
by a model of matter composed of 
tiny particles that are in motion. When 
substances undergo changes of state, 
neither atoms nor molecules them- 
selves are changed in structure. 

Mass is conserved when substances 
undergo changes of state. 


P8.7: Chemical changes can occur 
when two substances, elements, or 
compounds react and produce one or 
more different substances whose 
physical and chemical properties are 
different from the reacting substances. 
When substances undergo chemical 
change, the number and kinds of 
atoms in the reactants are the same as 
the number and kinds of atoms in the 
products. Mass is conserved when 
substances undergo chemical change. 
The mass of the reactants is the same 
as the mass of the products. 


P12.5: Changes of state require a 
transfer of energy. Water has a very 
high specific heat, meaning it can 
absorb a large amount of energy while 
producing only small changes in 
temperature. 


P12.6: An atom’s electron 
configuration, particularly of the 
outermost electrons, determines how 
the atom can interact with other 
atoms. The interactions between 
atoms that hold them together in 
molecules or between oppositely 
charged ions are called chemical 
bonds. 


P12.7: A large number of important 
reactions involve the transfer of either 
electrons (oxidation/reduction 
reactions) or hydrogen ions (acid/base 
reactions) between reacting ions, 
molecules, or atoms. In other 
chemical reactions, atoms interact 
with one another by sharing electrons 
to create a bond. An important 
example is carbon atoms, which can 
bond to one another in chains, rings, 
and branching networks to form, 
along with other kinds of atoms 
(hydrogen, oxygen, nitrogen, and 
sulfur), a variety of structures, 
including synthetic polymers, oils, 
and the large molecules essential to 
life. 


* See textbox on p. 28 for more information on the unique properties of water. 
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Grade 4 


Exhibit 8 (continued). Ph 


sical Science content statements for 


rades 4, 8, and 12 
Grade 12 


P4.7: Heat (thermal energy), 
electricity, light, and sound are forms 
of energy.® 


P4.8: Heat (thermal energy) results 
when substances burn, when certain 
kinds of materials rub against each 
other, and when electricity flows 
though wires. Metals are good 
conductors of heat (thermal energy) 
and electricity. Increasing the 
temperature of any substance requires 
the addition of energy. 


P4.9: Light travels in straight lines. 
When light strikes substances and 
objects through which it cannot pass, 
shadows result. When light travels 
obliquely from one substance to 
another (air and water), it changes 
direction. 


P4.10: Vibrating objects produce 
sound. The pitch of sound can be 
varied by changing the rate of 
vibration. 


P8.8: Objects and substances in 
motion have kinetic energy. For 
example, a moving baseball can break 
a window; water flowing down a 
stream moves pebbles and floating 
objects along with it. 


P8.9: Three forms of potential energy 
are gravitational, elastic, and 
chemical. Gravitational potential 
energy changes in a system as the 
relative positions of objects are 
changed. Objects can have elastic 
potential energy due to their 
compression, or chemical potential 
energy due to the nature and 
arrangement of the atoms. 


P8.10: Energy is transferred from 
place to place. Light energy from the 
Sun travels through space to Earth 
(radiation). Thermal energy travels 
from a flame through the metal of a 
cooking pan to the water in the pan 
(conduction). Air warmed by a 
fireplace moves around a room 
(convection). Waves (including sound 
and seismic waves, waves on water, 
and light waves) have energy and 
transfer energy when they interact 
with matter. 


P8.11: A tiny fraction of the light 
energy from the Sun reaches Earth. 
Light energy from the Sun is Earth’s 
primary source of energy, heating 
Earth surfaces and providing the 
energy that results in wind, ocean 
currents, and storms. 


P12.8: Atoms and molecules that 
compose matter are in constant 
motion (translational, rotational, or 
vibrational). 


P12.9: Energy may be transferred 
from one object to another during 
collisions. 


P12.10: Electromagnetic waves are 
produced by changing the motion of 
charges or by changing magnetic 
fields. The energy of electromagnetic 
waves is transferred to matter in 
packets. The energy content of the 
packets is directly proportional to the 
frequency of the electromagnetic 
waves. 


P12.11: Fission and fusion are 
reactions involving changes in the 
nuclei of atoms. Fission is the splitting 
of a large nucleus into smaller nuclei 
and particles. Fusion involves joining 
two relatively light nuclei at 
extremely high temperature and 
pressure. Fusion is the process 
responsible for the energy of the Sun 
and other stars. 


’ See footnote 5 on p. 29 for more information on the framework’s use of the terms “heat” and “thermal 


energy.” 
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Grade 4 


Exhibit 8 (continued). Ph 


sical Science content statements for 


rades 4, 8, and 12 
Grade 12 


Energy Transfer and Conservation: Electrical circuits (4); energy transfer and conservation of energy (8); 
translational, rotational, and vibrational energy of atoms and molecules, and chemical and nuclear reactions 
(12). 


P4.11: Electricity flowing through an 
electrical circuit produces magnetic 
effects in the wires. In an electrical 
circuit containing a battery, a bulb, 
and a bell, energy from the battery is 
transferred to the bulb and the bell, 
which in turn transfer the energy to 
their surroundings as light, sound, and 
heat (thermal energy). 


P8.12: When energy is transferred 
from one system to another, the 
quantity of energy before transfer 
equals the quantity of energy after 
transfer. For example, as an object 
falls, its potential energy decreases as 
its speed, and consequently, its kinetic 
energy increases. While an object is 
falling, some of the object’s kinetic 
energy is transferred to the medium 
through which it falls, setting the 
medium into motion and heating it. 


P8.13: Nuclear reactions take place in 
the Sun. In plants, light from the Sun 
is transferred to oxygen and carbon 
compounds, which, in combination, 
have chemical potential energy 
(photosynthesis). 


P12.12: Heating increases the 
translational, rotational, and 
vibrational energy of the atoms 
composing elements and the 
molecules or ions composing 
compounds. As the translational 
energy of the atoms, molecules, or 
ions increases, the temperature of the 
matter increases. Heating a sample of 
a crystalline solid increases the 
vibrational energy of the atoms, 
molecules, or ions. When the 
vibrational energy becomes great 
enough, the crystalline structure 
breaks down and the solid melts. 


P12.13: The potential energy of an 
object on Earth’s surface is increased 
when the object’s position is changed 
from one closer to Earth’s surface to 
one farther from Earth’s surface. 


P12.14: Chemical reactions either 
release energy to the environment 
(exothermic) or absorb energy from 
the environment (endothermic). 


P12.15: Nuclear reactions (fission and 
fusion) convert very small amounts of 
matter into appreciable amounts of 
energy. 


P12.16: Total energy is conserved ina 
closed system. 
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Exhibit 8 (continued). Ph 


sical Science content statements for 


rades 4, 8, and 12 


P4.12: An object’s position can be 
described by locating the object 
relative to other objects or a 
background. The description of an 
object’s motion from one observer’s 
view may be different from that 
reported from a different observer’s 
view. 


P4.13: An object is in motion when 
its position is changing. The speed of 
an object is defined by how far it 
travels divided by the amount of time 
it took to travel that far. 


P8.14: An object’s motion can be 
described by its speed and the 
direction in which it is moving. An 
object’s position can be measured and 
graphed as a function of time. An 
object’s speed can be measured and 
graphed as a function of time. 


P12.17: The motion of an object can 
be described by its position and 
velocity as functions of time and by 
its average speed and average 
acceleration during intervals of time. 


P12.18: Objects undergo different 
kinds of motion (translational, 
rotational, and vibrational). 
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Exhibit 8 (continued). Ph 
Grade 4 


sical Science content statements for 


rades 4, 8, and 12 
Grade 12 


Forces Affecting Motion: The association of changes in motion with forces and the association of objects 
falling toward Earth with gravitational force (4); qualitative descriptions of magnitude and direction as 
characteristics of forces, addition of forces, contact forces, forces that act at a distance, and net force on an 
object and its relationship to the object’s motion (8); quantitative descriptions of universal gravitational and 
electric forces, and relationships among force, mass, and acceleration (12). 


P4.14: The motion of objects can be 
changed by pushing or pulling. The 
size of the change is related to the size 
of the force (push or pull) and the 
weight (mass) of the object on which 
the force is exerted. When an object 
does not move in response to a push 
or a pull, it is because another push or 
pull (friction) is being applied by the 
environment. 


P4.15: Earth pulls down on all objects 
with a force called gravity. With a few 
exceptions (helium-filled balloons), 
objects fall to the ground no matter 
where the object is on Earth. 


P8.15: Some forces between objects 
act when the objects are in direct 
contact or when they are not touching. 
Magnetic, electrical, and gravitational 
forces can act at a distance. 


P8.16: Forces have magnitude and 
direction. Forces can be added. The 
net force on an object is the sum of all 
the forces acting on the object. A 
nonzero net force on an object 
changes the object’s motion; that is, 
the object’s speed and/or direction of 
motion changes. A net force of zero 
on an object does not change the 
object’s motion; that is, the object 
remains at rest or continues to move at 
a constant speed in a straight line. 


P12.19: The motion of an object 
changes only when a net force is 
applied. 


P12.20: The magnitude of 
acceleration of an object depends 
directly on the strength of the net 
force and inversely on the mass of the 
object. This relationship (a=F,,¢,/m) is 
independent of the nature of the force. 


P12.21: Whenever one object exerts 
force on another, a force equal in 
magnitude and opposite in direction is 
exerted by the second object back on 
the first object. In closed systems, 
momentum is the quantity of motion 
that is conserved. Conservation of 
momentum can be used to help 
validate the relationship a=F,,.//m. 


P12.22: Gravitation is a universal 
attractive force that each mass exerts 
on any other mass. The strength of the 
gravitational force between two 
masses is proportional to the masses 
and inversely proportional to the 
square of the distance between them. 


P12.23: Electric force is a universal 
force that exists between any two 
charged objects. Opposite charges 
attract while like charges repel. The 
strength of the electric force is 
proportional to the magnitudes of the 
charges and inversely proportional to 
the square of the distance between 
them. Between any two charged 
particles, the electric force is vastly 
greater than the gravitational force. 
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LIFE SCIENCE 


Life Science principles are essential for understanding the functioning of living organ- 
isms and their interactions with their environment. In addition, Life Science principles are 
crucial for understanding advances in science and technology and appreciating their 
implications for social and personal decisions. For example, the following discoveries of 
the past 25 years rely on understanding basic ideas in life science: publication of the 
human genome and genomes of other organisms, ability to monitor the oxygen level of 
specific regions of the brain, and depletion of the ozone layer by human activities. The 
media regularly ask questions related to health and disease, such as what constitutes a 
healthy lifestyle and how to deal with the mutability of bacteria, viruses, and parasites 
that thwart efforts to develop antibiotics and vaccines. Although science does not current- 
ly provide complete answers to these types of questions, it provides the tools for under- 
standing and addressing them. 


Understanding principles in Life Science is inextricably linked with understanding prin- 
ciples in Physical Science and Earth and Space Sciences. “Living organisms are made of 
the same components as all other matter, involve the same kind of transformations of 
energy, and move using the same basic kinds of forces” (AAAS 1994, p. 59). 


Understanding living systems and their interactions with their environment requires not 
only an understanding of various levels of biological organization—molecules, cells, 
tissues/organs, organisms, populations, ecosystems—but also an understanding of inter- 
actions (including the transfer of information) within and across these levels and how 
they can change over time. For example, understanding how populations of organisms 
change over time is greatly facilitated by understanding the changes that occur in DNA 
molecules. These changes are manifest in an organism’s traits and may affect its ability to 
survive and reproduce, which can lead to changing proportions of traits in populations 
over time. 


As summarized in exhibit 9, the Life Science content statements are sorted into topics 
and subtopics that, collectively, address structure, function, and patterns of change in liv- 
ing systems. However, any attempt to organize Life Science by a linear set of topics and 
subtopics, such as those listed below, is somewhat arbitrary. The overlap is evident in 
exhibit 10, Life Science Content Statements for Grades 4, 8, and 12. 
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Exhibit 9. Life Science content topics and subtopics 
Structures and Functions of Living Systems 


Organization and development 
Matter and energy transformations 
Interdependence 


Changes in Living Systems 


Heredity and reproduction 
Evolution and diversity 


STRUCTURES AND FUNCTIONS OF LIVING SYSTEMS 


This topic comprises the ways in which living systems are organized and how living 
systems carry out their life functions. Reasoning about living systems often involves 
relating different levels of organization, from the molecule to the biosphere, and 
understanding how living systems are structured at each level. The functions of living 
systems at these levels, particularly how they transform matter and energy, are included, 
as are the interactions among living systems and how they depend on one another to carry 
out their functions. 


Organization and Development 


As it was pointed out early in the 20th century, “Long ago it became evident that the key 
to every biological problem must finally be sought in the cell; for every living organism 
is, or at some time has been, a cell” (Wilson 1928, p. 1). All living things are made up of 
cells whose work is carried out by many different types of molecules. Cellular and 
molecular biology has the power to explain a wide variety of phenomena related to the 
organization and development of living systems, such as synthesis and reproduction, the 
extraction of energy from food, and regulation. Living organisms have a variety of 
observable features that enable them to obtain food and reproduce (grade 4). The func- 
tions of living organisms are carried out at different levels of organization. In multi- 
cellular organisms, cells form organs and organ systems (grade 8). Organisms are subsys- 
tems of populations, communities, ecosystems, and the biosphere. Cellular processes are 
carried out by molecules, particularly proteins. These processes are regulated, both inter- 
nally and externally, by the environments in which cells exist, including local environ- 
ments that lead to cell differentiation during the development of multicellular organisms 
(grade 12). 
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Matter and Energy Transformations 


Matter and energy transformations are involved in all life processes, such as photosynthe- 
sis, growth and repair, cellular respiration, and the need of living systems for continual 
input of energy. 


All single-celled and multicellular organisms have the same basic needs: water, air, a 
source of energy and materials for growth and repair, waste disposal, and conditions for 
growth and reproduction (grade 4). In terms of matter and energy transformations, the 
source of food is the distinguishing difference between plants and animals (see textbox 
below). 


Clarification: Food 


Both plants and animals require a source of energy and materials for growth and repair, 
and both plants and animals use high-energy compounds as a source of fuel and building 
material. Plants are distinguished from animals by the fact that plants have the capability 
(through photosynthesis) to take energy from light to form higher energy molecules 
containing carbon, hydrogen, and oxygen (carbohydrates) from lower energy molecules. 


Plants are similar to animals in that, to make other molecules for their growth and 


reproduction, they use the energy that is released as carbohydrates react with oxygen. In 
making these other molecules, plants use the breakdown products of carbohydrates, along 
with minerals from the soil and from fertilizers (known colloquially as “plant foods”), as 
building blocks. Plants also synthesize substances (carbohydrates, fats, proteins, 
vitamins) that are components of foods eaten by animals. 


Therefore, although synthesis and breakdown are common to both plants and animals, 
photosynthesis (the conversion of light energy into stored chemical energy) is unique to 
plants, making them the primary source of energy for all animals. 


Basic needs are connected with the processes of growth and metabolism. Organisms are 
made up of carbon-containing molecules; these molecules originate in molecules that 
plants assemble from carbon dioxide and water. In converting carbon-containing mole- 
cules back to water and carbon dioxide, organisms release energy, making some of it 
available to support life functions (grade 8). Matter and energy transformations in cells, 
organisms, and ecosystems have a chemical basis (grade 12). The following textbox on 
the flow of energy through an ecosystem illustrates principles that cut across the content 
areas. 
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Crosscutting Content: Uses, Transformations, and Conservation of Energy 


The principles of energy uses, transformations, and conservation hold true across different types 
of systems. These systems include biological organisms, Earth systems, ecosystems (combining 
both life forms and their physical environment), the solar system, other systems in the universe, 
and human-designed systems. 


However complex the workings of living organisms, they share with all other 
natural systems the same physical principles of the conservation and 
transformation of matter and energy. Over long spans of time, matter and energy 
are transformed among living things, and between them and the physical 
environment. In these grand-scale cycles, the total amount of matter and energy 
remains constant, even though their form and location undergo continual change. 


Almost all life on earth is ultimately maintained by transformations of energy 
from the Sun. Plants capture the Sun’s energy and use it to synthesize complex, 
energy-rich molecules (chiefly sugars) from molecules of carbon dioxide and 
water. These synthesized molecules then serve, directly or indirectly, as the 
source of energy for the plants themselves and ultimately for all animals and 
decomposer organisms (such as bacteria and fungi). This is the food web: The 
organisms that consume the plants derive energy and materials from breaking 
down the plant molecules, use them to synthesize their own structures, and then 
are themselves consumed by other organisms. At each stage in the food web, 
some energy is stored in newly synthesized structures and some is dissipated into 
the environment as heat produced by the energy-releasing chemical processes in 
cells. A similar energy cycle begins in the oceans with the capture of the Sun’s 
energy by tiny, plant-like organisms. Each successive stage in a food web 
captures only a small fraction of the energy content of organisms it feeds on. 


Science for All Americans (AAAS 1994, p. 66). 


The flow of energy in an ecosystem (such as that described above) can be compared to the flow 
of energy illustrated earlier (see “Crosscutting Content: Energy Sources and Transfer” textbox on 
p. 31). They are both identical (the principle) and different (the context). In each case, energy is 
transformed from one form to another; while some is no longer available for human use, it is not 
lost to the system. 


The following grade 4 content statements illustrate the crosscutting nature of uses, 
transformations, and conservation of energy. They are not intended to represent an exhaustive 
catalog of all statements related to this crosscutting content. 


Physical Science Life Science Earth and Space Sciences 


P4.7: Heat (thermal energy), L4.2: Organisms have basic E4.7: The Sun warms the land, 
electricity, light, and sound are needs. Animals require air, air, and water and helps plants 
forms of energy. water, and a source of energy row. 

and building material for growth 
and repair. Plants also require 
light. 
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Interdependence 


The interaction of species in an ecosystem, the dynamics of population growth and 
decline, the use of resources by multiple species, the impact of species on their environ- 
ment, and the complex interactions among all of these forces have enormous conse- 
quences for the survival of all species, including humans. 


All animals and most plants depend on both other organisms and their environments for 
their basic needs (grade 4). Organisms interact with one another in a variety of ways, 
such as producer/consumer, predator/prey, and parasite/host. In addition to competition 
among organisms, the size of populations depends on environmental conditions such as 
the availability of water, light, and other suitable conditions (grade 8). Ecosystems are 
characterized by both stability and change, on which human populations can have an 
impact (grade 12). 


All animals and most plants depend on both other organisms and their environments for 
their basic needs (grade 4). Organisms interact with one another in a variety of ways, 
such as producer/consumer, predator/prey, and parasite/host. In addition to competition 
among organisms, the size of populations depends on environmental conditions such as 
the availability of water, light, and other suitable conditions (grade 8). Ecosystems are 
characterized by both stability and change, on which human populations can have an 
impact (grade 12). 


CHANGES IN LIVING SYSTEMS 


This topic comprises how organisms reproduce, how they pass genetic information to 
their offspring, and how genetic information can change as it passes from one generation 
to the next. Over time, these changes can affect the size, diversity, and genetic composi- 
tion of populations (i.e., the process of biological evolution). 


Heredity and Reproduction 


Organisms closely resemble their parents; their slight variations can accumulate over 
many generations and result in more obvious differences between organisms and their 
ancestors. Recent advances in biochemistry and cell biology have increased understand- 
ing of the mechanisms of inheritance and have enabled the detection of disease-related 
genes. Such knowledge is making it possible to design and produce large quantities of 
substances to treat disease and, in years to come, may lead to cures. 


All plants and animals (and one-celled organisms) develop and have the capacity to 
reproduce (grade 4). Reproduction, whether sexual or asexual, is a requirement for the 
survival of a species. Characteristics of organisms are influenced by heredity and envi- 
ronment (grade 8). Genetic differences among individuals and species are fundamentally 
chemical. Different organisms are made up of somewhat different proteins. Reproduction 
involves passing the DNA with instructions for making these proteins from one genera- 
tion to the next with occasional modifications (grade 12). 
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Evolution and Diversity 


Earth’s present-day life forms have evolved from common ancestors reaching back to the 
simplest one-celled organisms almost 4 billion years ago. Modern ideas about evolution 
provide a scientific explanation for three main sets of observable facts about life on 
Earth: the enormous number of different life forms that exist, the systematic similarities 
in anatomy and molecular chemistry seen within that diversity, and the sequences of 
changes in fossils found in successive layers of rock that have been formed over more 
than a billion years. The modern concept of evolution, including natural selection and 
common descent, provides a unifying principle for understanding the history of life on 
Earth, relationships among all living things, and the dependence of life on the physical 
environment. The concept is so well established that it provides a framework for organiz- 
ing most of biological knowledge into a coherent picture. 


All organisms are similar to and different from other organisms, and some kinds of or- 
ganisms and individuals have advantages in particular environments (grade 4). Preferen- 
tial survival means that differences among individuals in a population affect their ability 
to survive and reproduce. Classification reflects degrees of relatedness among species 
(grade 8). Evolution is the consequence of natural selection and differential reproduction. 
Natural selection and common descent provide the scientific explanation for the history 
of life on Earth as depicted in the fossil record and as indicated by anatomical and chemi- 
cal similarities evident within the diversity of existing organisms (grade 12). 
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Grade 4 


Exhibit 10. Life Science content statements for 


Structures and Functions of Living Systems 
Organization and Development: Basic needs of organisms (4), levels of organization of living 


systems (8) the chemical basis of living systems (12). 


rades 4, 8, and 12 


Grade 12 


L4.1: Organisms need food, water, 
and air; a way to dispose of waste; 
and an environment in which they can 
live.* 


L8.1: All organisms are composed of 
cells, from one cell only to many 
cells. About two-thirds of the weight 
of cells is accounted for by water, 
which gives cells many of their 
properties. In multicellular organisms, 
specialized cells perform specialized 
functions. Organs and organ systems 
are composed of cells and function to 
serve the needs of cells for food, air, 
and waste removal. The way in which 
cells function is similar in all living 
organisms.! 


L8.2: Following fertilization, cell 
division produces a small cluster of 
cells that then differentiate by 
appearance and function to form the 
basic tissues of an embryo. 


L12.1: Living systems are made of 
complex molecules (including 
carbohydrates, fats, proteins, and 
nucleic acids) that consist mostly of a 
few elements, especially carbon, 
hydrogen, oxygen, nitrogen, and 
phosphorous. 


L12.2: Cellular processes are carried 
out by many different types of 
molecules, mostly proteins. Protein 
molecules are long, usually folded 
chains made from combinations of 
amino-acid molecules. Protein 
molecules assemble fats and 
carbohydrates and carry out other 
cellular functions. The function of 
each protein molecule depends on its 
specific sequence of amino acids and 
the shape of the molecule. 


L.12.3: Cellular processes are 
regulated both internally and 
externally by environments in which 
cells exist, including local 
environments that lead to cell 
differentiation during the development 
of multicellular organisms. During the 
development of complex multicellular 
organisms, cell differentiation is 
regulated through the expression of 
different genes. 


* See p. 41 for a textbox on “Food.” 
* Human organs and organ systems are subsumed under this content statement. See the specifications for 


more information. 
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Grade 4 


Exhibit 10 (continued). Life Science content statements for 


Structures and Functions of Living Systems 


Matter and Energy Transformations: The basic needs of organisms for growth (4), the role of 
carbon compounds in growth and metabolism (8), the chemical basis of matter and energy 


transformation in living systems (12). 


rades 4, 8, and 12 
Grade 12 


L4.2: Organisms have basic needs. 
Animals require air, water, and a 
source of energy and building material 
for growth and repair. Plants also 
require light. 


L8.3: Cells carry out the many 
functions needed to sustain life. They 
grow and divide, thereby producing 
more cells. Food is used to provide 
energy for the work that cells do and 
is a source of the molecular building 
blocks from which needed materials 
are assembled. 


L8.4: Plants are producers; that is, 
they use the energy from light to make 
sugar molecules from the atoms of 
carbon dioxide and water.* Plants use 
these sugars along with minerals from 
the soil to form fats, proteins, and 
carbohydrates. These products can be 
used immediately, incorporated into 
the plant’s cells as the plant grows, or 
stored for later use. 


L8.5: All animals, including humans, 
are consumers that meet their energy 
needs by eating other organisms or 
their products. Consumers break down 
the structures of the organisms they 
eat to make the materials they need to 
grow and function. Decomposers, 
including bacteria and fungi, use dead 
organisms or their products to meet 
their energy needs. 


L12.4: Plants have the capability 
(through photosynthesis) to take 
energy from light to form higher 
energy sugar molecules containing 
carbon, hydrogen, and oxygen from 
lower energy molecules. These sugar 
molecules can be used to make amino 
acids and other carbon-containing 
(organic) molecules and assembled 
into larger molecules with biological 
activity (including proteins, DNA, 
carbohydrates, and fats). 


L12.5: The chemical elements that 
make up the molecules of living 
things pass through food webs and are 
combined and recombined in different 
ways. At each link in an ecosystem, 
some energy is stored in newly made 
structures, but much is dissipated into 
the environment as heat. Continual 
input of energy from sunlight keeps 
the process going. 


L12.6: As matter cycles and energy 
flows through different levels of 
organization of living systems (cells, 
organs, organisms, communities) and 
between living systems and the 
physical environment, chemical 
elements are recombined in different 
ways. Each recombination results in 
storage and dissipation of energy into 
the environment as heat. Matter and 
energy are conserved in each change. 


* The statement “they use the energy from light” does not imply that energy is converted into matter or that 
energy is lost. See textbox “Crosscutting Content: Uses, Transformations, and Conservation of Energy.” 


NAEP 2015 SCIENCE FRAMEWORK 
43 


Grade 4 


Exhibit 10 (continued). Life Science content statements for 
Structures and Functions of Living Systems 
Interdependence: The interdependence of organisms (4), specific types of interdependence (8), 
consequences of interdependence (12). 


rades 4, 8, and 12 
Grade 12 


L4.3: Organisms interact and are 
interdependent in various ways, 
including providing food and shelter 
to one another. Organisms can survive 
only in environments in which their 
needs are met. Some interactions are 
beneficial; others are detrimental to 
the organism and other organisms. 


L4.4:; When the environment changes, 
some plants and animals survive and 
reproduce; others die or move to new 
locations. 


L8.6: Two types of organisms may 
interact with one another in several 
ways: They may be in a producer/ 
consumer, predator/prey, or parasite/ 
host relationship. Or, one organism 
may scavenge or decompose another. 
Relationships may be competitive or 
mutually beneficial. Some species 
have become so adapted to each other 
that neither could survive without the 
other. 


L8.7: The number of organisms and 
populations an ecosystem can support 
depends on the biotic resources 
available and abiotic factors, such as 
quantity of light and water, range of 
temperatures, and soil composition. 


L8.8: All organisms cause changes in 
the environment where they live. 
Some of these changes are detrimental 
to the organisms or other organisms, 
whereas others are beneficial. 


L12.7: Although the interrelationships 
and interdependence of organisms 
may generate biological communities 
in ecosystems that are stable for 
hundreds or thousands of years, 
ecosystems always change when 
climate changes or when one or more 
new species appear as a result of 
migration or local evolution. The 
impact of the human species has 
major consequences for other species. 
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Exhibit 10 (continued). Life Science content statements for grades 4, 8, and 12 


Grade 4 
Changes in Living Systems 


Grade 8 


Grade 12 


Heredity and Reproduction: Life cycles (4), reproduction and the influence of heredity and the 
environment on an offspring’s characteristics (8), the molecular basis of heredity (12). 


L4.5: Plants and animals have life 
cycles. Both plants and animals begin 
life and develop into adults, 
reproduce, and eventually die. The 
details of this life cycle are different 
for different organisms. 


L4.6: Plants and animals closely 
resemble their parents. 


L8.9: Reproduction is a characteristic 
of all living systems; because no 
individual organism lives forever, 
reproduction is essential to the 
continuation of every species. Some 
organisms reproduce asexually. Other 
organisms reproduce sexually. 


L8.10: The characteristics of 
organisms are influenced by heredity 
and environment. For some 
characteristics, inheritance is more 
important; for other characteristics, 
interactions with the environment are 
more important. 


L12.8: Hereditary information is 
contained in genes, which are located 
in the chromosomes of each cell. A 
human cell contains many thousands 
of different genes. One or many genes 
can determine an inherited trait of an 
individual, and a single gene can 
influence more than one trait. 


L12.9: The genetic information 
encoded in DNA molecules provides 
instructions for assembling protein 
molecules. Genes are segments of 
DNA molecules. Inserting, deleting, 
or substituting DNA segments can 
alter genes. An altered gene may be 
passed on to every cell that develops 
from it. The resulting features may 
help, harm, or have little or no effect 
on the offspring’s success in its 
environment. 


L12.10: Sorting and recombination of 
genes in sexual reproduction results in 
a great variety of possible gene 
combinations from the offspring of 
any two parents. 
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Grade 4 
Changes in Living Systems 


Exhibit 10 (continued). Life Science content statements for 


rades 4, 8, and 12 
Grade 12 


Evolution and Diversity: Differences and adaptations of organisms (4), preferential survival and 
relatedness of organisms (8), the mechanisms of evolutionary change and the history of life on Earth 


(12). 


L4.7: Different kinds of organisms 
have characteristics that enable them 
to survive in different environments. 
Individuals of the same kind differ in 
their characteristics, and sometimes 
the differences give individuals an 
advantage in surviving and 
reproducing. 


L8.11: Individual organisms with 
certain traits in particular 
environments are more likely than 
others to survive and have offspring. 
When an environment changes, the 
advantage or disadvantage of 
characteristics can change. Extinction 
of a species occurs when the 
environment changes and the 
characteristics of a species are 
insufficient to allow survival. Fossils 
indicate that many organisms that 
lived long ago are extinct. Extinction 
of a species is common; most of the 
species that have lived on the Earth no 
longer exist. 


L8.12: Similarities among organisms 
are found in anatomical features, 
which can be used to infer the degree 
of relatedness among organisms. In 
classifying organisms, biologists 
consider details of internal and 
external structures to be more 
important than behavior or general 
appearance. 


L12.11: Modern ideas about evolution 
(including natural selection and 
common descent) provide a scientific 
explanation for the history of life on 
Earth as depicted in the fossil record 
and in the similarities evident within 
the diversity of existing organisms. 


L12.12: Molecular evidence 
substantiates the anatomical evidence 
for evolution and provides additional 
detail about the sequence in which 
various lines of descent branched. 


L12.13: Evolution is the consequence 
of the interactions of (1) the potential 
for a species to increase its numbers, 
(2) the genetic variability of offspring 
due to mutation and recombination of 
genes, (3) a finite supply of the 
resources required for life, and (4) the 
ensuing selection from environmental 
pressure of those organisms better 
able to survive and leave offspring. 
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EARTH AND SPACE SCIENCES 


The past few decades have brought rapid changes in the character of Earth and Space 
Sciences. The study of Earth has shifted from surface geology and mining toward global 
change and Earth systems; research methods have changed from human observations and 
mapping to remote sensing and computer modeling.° This concept of Earth as a complex 
and dynamic entity of interrelated subsystems implies that there is no process or phenom- 
enon within the Earth system that occurs in complete isolation from other elements of the 
system. There has also been a shift in goals as advances in theory have made it possible 
to more accurately predict changes in weather and climate; to provide life-saving warn- 
ings of floods, hurricanes, earthquakes, and volcanic eruptions; and to understand how 
human activities influence ecosystem and climate changes across the globe. 


In space science, similar changes have taken place as a result of new technologies. 
Successful probes to Mars, Jupiter, and Saturn have vastly expanded knowledge of the 
solar neighborhood. The discovery of more than 100 planets outside the solar system has 
raised new questions about the origin of life. Furthermore, advances in ground and space- 
based telescopes capable of observing many different parts of the electromagnetic 
spectrum with unprecedented detail have revolutionized understanding of the structure 
and evolution of the universe itself. In brief, descriptive methods of Earth and Space 
Sciences have given way to theory-based inquiry and problem-solving approaches that 
have far-reaching consequences with regard to understanding the universe and steward- 
ship of planet Earth. 


Changes in Earth and Space Science education are beginning to catch up with advances 
in research. The National Standards emphasize a systems approach to studying Earth, 
especially at the high school level. Some of today’s textbooks pay less attention to 
describing Earth features and focus instead on a systems perspective in which Earth is 
viewed as a physical system of interrelated phenomena, processes, and cycles. Some high 
school curricula have integrated the traditional Earth science disciplines of geology, 
meteorology, and oceanography with aspects of biology, chemistry, and physics to intro- 
duce students to a more holistic study of Earth. 


The tools available to students for learning about Earth and space have changed as well, 
although not all of these resources are available to all students. Visualization tools, such 
as geographical information system (GIS) software, have made it possible for Earth 
science students to have direct access to the raw data and models used by scientists. Other 
Web-based programs allow students to view and process satellite images of Earth, to 
direct a camera on board the Space Shuttle, and to access professional telescopes around 
the world to carry out science projects. In other words, the core concepts, subject matter, 
and tools used by students have undergone profound changes in recent decades that 


° “Farth” is capitalized, rather than referred to as “the earth,” in order to recognize it as one of the planets in 
the solar system. 


NAEP 2015 SCIENCE FRAMEWORK 
47 


mirror many of the advances in Earth and Space Sciences. The data and images gathered 
by these tools could be used as source materials for assessment items. For example, 
students could examine data on sea surface temperatures and upper atmospheric winds, 
derived from satellite observations, to predict the intensity and track of a hurricane. 


To reflect the importance of this content area, NAEP will include questions about Earth 
and Space Sciences at the 4th-, 8th-, and 12th-grade levels. The content statements have 


been divided into topics and subtopics as summarized in exhibit 11. 


Exhibit 11. Earth and Space Sciences content topics and subtopics 


Earth in Space and Time 


Objects in the universe 
History of Earth 


Earth Structures 


Properties of Earth materials 
Tectonics 


Earth Systems 


Energy in Earth systems 
Climate and weather 
Biogeochemical cycles 


EARTH IN SPACE AND TIME 


Earth in space and time is divided into two subtopics: objects in the universe and history 
of Earth. The idea that “the universe is large and ancient, on scales staggering to the 
human mind” (AAAS 1994, p. 40) connects these subtopics. 


One of the earliest discoveries of the scientific age was that Earth is not the center of the 
universe. It is now known that Earth is a planet in space, one of a family of planets and 
other bodies that circle a yellow star in a vast galaxy of other stars. Like countless other 
worlds that are known to exist, Earth has a beginning and a history. That history can be 
read by carefully and thoughtfully observing the world and the universe. 


Objects in the Universe 


Objects in the sky, such as the Sun and the Moon, have patterns of movement. These 
patterns can be observed through changes in shape or placement in the sky based on time 
of day or season (grade 4). By recognizing these patterns, people have developed 
calendars and clocks and explained such phenomena as Moon phases, eclipses, and 
seasons (grade 8). 
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It was previously thought that Earth was the center of the universe. However, it is now 
known that the Sun is the central and largest body in the solar system, which includes 
Earth and other planets and their moons as well as other objects such as asteroids and 
comets. Objects in the solar system are kept in predictable motion by the force of gravity 
(grade 8). 


According to the “big bang” theory, the entire contents of the known universe expanded 
explosively into existence from a hot, dense state 13.7 billion years ago. Early in the his- 
tory of the universe, stars coalesced out of clouds of hydrogen and helium and clumped 
together by gravitational attraction into billions of galaxies. When heated to a sufficiently 
high temperature by gravitational attraction, stars begin nuclear reactions, which convert 
matter to energy and fuse light elements into heavier ones (grade 12). 


History of Earth 


Theories of planet formation and radioactive dating of meteorites have led to the conclu- 
sion that the Sun, Earth, and the rest of the solar system formed from a nebular cloud of 
dust and gas 4.6 billion years ago. Early Earth was very different from today’s planet. 
Initially, there was no life and no molecular oxygen in the atmosphere. Evidence shows 
that one-celled organisms (bacteria) were the first forms of life on the planet, appearing 
about 3.5 billion years ago. These bacteria are thought to be responsible for adding 
oxygen to Earth’s atmosphere, making it possible for a wider diversity of life forms to 
evolve (grade 12). 


Earth processes seen today, such as erosion and mountain building, have made possible 
the measurement of geologic time through methods such as observing rock sequences and 
using fossils to correlate the sequences at various locations. Fossils also provide evidence 
of how life and environmental conditions have changed (grade 8). Early methods of 
determining geological time, such as the use of index fossils and stratigraphic sequences, 
allowed for the relative dating of geologic events. However, absolute dating was impos- 
sible until the discovery that certain radioactive isotopes in rocks have known decay 
rates, making it possible to determine how many years ago a given rock sample formed 
(grade 12). 


Earth’s surface changes over time. Some changes are due to slow processes, such as 
erosion and weathering, and others are due to rapid processes such as volcanic eruptions, 
landslides, and earthquakes (grade 4). Changes caused by violent earthquakes and 
volcanic eruptions can be observed on a human time scale; on the other hand, many 
geological processes, such as the building of mountain chains and shifting of entire conti- 
nents, take place over hundreds of millions of years. Water, ice, waves, and wind sculpt 
Earth’s surface to produce distinctive landforms (grade 12). 


EARTH STRUCTURES 


Content statements related to Earth structures are divided into two subtopics: properties 
of Earth materials and tectonics. The study of Earth materials has contributed to under- 
standing dynamic processes, which are in turn driven by the movement of vast tectonic 
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plates. Conversely, the development of tectonic theory has made it possible to locate and 
extract Earth materials for a wide variety of human uses. 


Properties of Earth Materials 


Earth materials that occur in nature include rocks, minerals, soils, water, and the gases of 
the atmosphere. Natural materials have different properties that sustain plant and animal 
life (grade 4). Soil consists of weathered rocks and decomposed organic material from 
dead plants, animals, and bacteria. Soils are often found in layers, with each having a dif- 
ferent chemical composition and texture (grade 8). Some Earth materials have properties 
either in their present form or after design and modification that make them useful in 
solving human problems and enhancing the quality of life (grade 4). 


Rocks and rock formations bear evidence of the conditions and forces that created them, 
ranging from the violent conditions of volcanic eruptions to the slow deposition of sedi- 
ments. The atmosphere is a mixture of nitrogen, oxygen, and trace gases that include 
water vapor. The atmosphere has a different physical and chemical composition at differ- 
ent elevations (grade 8). 


Tectonics 


A basic understanding of geological history (described above) forms the foundation for 
later understanding of tectonics. Earth’s internal structure is layered with a lithosphere, 
hot convecting mantle, and dense metallic core. Lithospheric plates, on the scale of conti- 
nents and oceans, constantly move at rates of centimeters per year in response to move- 
ments in the mantle. Major geological events such as earthquakes, volcanic eruptions, 
and mountain building result from these plate motions (grade 8). 


Although continental drift was first suggested as early as the late 16th century and further 
developed in the early 1900s, it was not widely accepted until more convincing evidence 
emerged as a result of extensive exploration of the sea floor. By the late 1960s, mapping 
of the Mid-Atlantic Ridge, evidence of sea floor spreading, and subduction led to the 
more general theory of plate tectonics. The current explanation is that the outward trans- 
fer of Earth’s internal heat propels the plates comprising Earth’s surface across the face 
of the globe, pushing the plates apart where magma rises to form mid-ocean ridges, and 
pulling the edges of plates back down where Earth materials sink into the crust at deep 
trenches (grade 12). 


Earth as a whole has a magnetic field that is detectable at the surface with a compass. 
Earth’s magnetic field is similar to the field of a natural or manmade magnet with north 
and south poles and lines of force. For thousands of years, people have used compasses to 
aid navigation on land and sea (grade 8). Crucial evidence in support of tectonic theory 
came from studies of the magnetic properties of rocks on the ocean floor (grade 12). 
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EARTH SYSTEMS 


Earth systems is divided into three subtopics: energy in Earth systems, climate and 
weather, and biogeochemical cycles. The explorers of the 16th century who circumnavi- 
gated the planet were the first to become aware of global weather and climate patterns. 
As science began to mature and diversify in the 19th and 20th centuries, those who 
studied the planet scientifically did so from the perspective of the traditional disciplines, 
such as geology, oceanography, and meteorology. Currently, working with vastly 
improved technologies, most scientists take an Earth systems perspective, including the 
study of how energy moves through Earth systems and the integration of disciplines to 
better understand Earth’s biogeochemical cycles. 


Energy in Earth Systems 


The Sun warms the land, air, and water and helps plants grow (grade 4). The Sun is the 
major source of energy for phenomena on Earth’s surface. It drives convection within the 
atmosphere and oceans, producing winds, ocean currents, and the water cycle. Seasons 
result from annual variations in the intensity of sunlight and length of day due to the tilt 
of Earth’s rotation axis relative to the plane of its yearly orbit around the Sun (grade 8). 


Earth’s systems have internal and external sources of energy, both of which create heat. 
The Sun is the major external source of energy. Two primary sources of internal energy 
are the decay of radioactive isotopes and the gravitational/thermal energy from Earth’s 
original formation (grade 12). 


Climate and Weather 


Weather changes from day to day and during the seasons. Scientists use tools for record- 
ing and predicting weather changes (grade 4). Global patterns of atmospheric movement 
influence local weather (grade 8). 


Climate is determined by energy transfer from the Sun at and near Earth’s surface. This 
energy transfer is influenced by dynamic processes such as cloud cover, atmospheric gas- 
es, and Earth’s rotation, as well as static conditions such as the position of mountain 
ranges and oceans, seas, and lakes (grade 12). Oceans have a major effect on climate 
because water in the oceans holds a large amount of heat (grade 8). 


Biogeochemical Cycles 


Earth is a system containing essentially a fixed amount of each stable chemical atom or 
element. Each element can exist in several different chemical forms. Elements move 
within and between the lithosphere, atmosphere, hydrosphere, and biosphere as part of 
biogeochemical cycles (see the following textbox). Movement of matter through Earth’s 
systems is driven by Earth’s internal and external sources of energy. These movements 
are often accompanied by a change in the physical and chemical properties of the matter. 
For example, carbon occurs in carbonate rocks such as limestone, in coal and other fossil 
fuels, in the atmosphere as carbon dioxide gas, in water as dissolved carbon dioxide, and 
in all organisms as complex molecules that control the chemistry of life (grade 12). 
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Water, which covers the majority of Earth’s surface, circulates through the crust, oceans, 
and atmosphere in what is known as the water cycle. Water evaporates from Earth’s 
surface, rises and cools as it moves to higher elevations, condenses as clouds, falls as rain 
or snow, and collects in lakes, oceans, soil, and underground (grade 8). 


Natural ecosystems provide an array of basic processes that affect humans. These 
processes include maintenance of the quality of the atmosphere, generation of soils, 
control of the hydrologic cycle, disposal of wastes, and recycling of nutrients (grade 12). 
The supply of many Earth resources such as fuels, metals, fresh water, and farmland is 
limited. 


Humans change environments in ways that can either be beneficial or detrimental for 
themselves and other organisms (grade 4). Humans have devised methods for extending 
the use of Earth resources through recycling, reuse, and renewal (grade 4). However, oth- 
er activities (reducing the amount of forest cover, increasing the amount and variety of 
chemicals released into the atmosphere, and intensive farming) have changed Earth’s 
land, oceans, and atmosphere. Studies of plant and animal populations have shown that 
such activities can reduce the number and variety of wild plants and animals and some- 
times result in the extinction of species (grade 8). 
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Crosscutting Content: Biogeochemical Cycles 


To demonstrate an understanding of biogeochemical cycles, students must draw on their 
knowledge of matter and energy (Physical Science), structures and functions of living systems 
(Life Science), and Earth systems (Earth and Space Sciences). 


Essentially fixed amounts of chemical atoms or elements cycle within the Earth system, and 
energy drives their translocation and transformation. Examples of biogeochemical cycles include 
water, carbon, and nitrogen. The basic processes underlying the translocation of matter (e.g., 
changes of state, gravity) and transformations involving the rearrangement of atoms in chemical 
reactions are described in Physical Science (exhibit 8 on p. 34) and the role of living organisms in 
cycling atoms between inorganic and organic forms is described in Life Science (exhibit 10 on p. 
46). 


Biogeochemical cycles are described more fully in the Earth Systems section of exhibit 12, Earth 
and Space Science Content Statements for Grades 4, 8, and 12. 


The following grade 12 content statements illustrate the crosscutting nature of biogeochemical 
cycles. They are not intended to represent an exhaustive catalog of all statements related to this 


crosscutting content. 


Physical Science 


Life Science 


Earth and Space Sciences 


P12.7: A large number of 
important reactions involve the 
transfer of either electrons 
(oxidation/reduction reactions) 
or hydrogen ions (acid/base 
reactions) between reacting ions, 
molecules, or atoms. In other 
chemical reactions, atoms 
interact with one another by 
sharing electrons to create a 
bond. An important example is 
carbon atoms, which can bond to 
one another in chains, rings, and 
branching networks to form, 
along with other kinds of atoms 
(hydrogen, oxygen, nitrogen, 
and sulfur) a variety of 
structures, including synthetic 
polymers, oils, and the large 
molecules essential to life. 


L12.5: The chemical elements 
that make up the molecules of 
living things pass through food 
webs and are combined and 
recombined in different ways. At 
each link in an ecosystem, some 
energy is stored in newly made 
structures, but much is 
dissipated into the environment 
as heat. Continual input of 
energy from sunlight keeps the 
process going. 


E12.12: Movement of matter 
through Earth’s systems is 
driven by Earth’s internal and 
external sources of energy. 
These movements are often 
accompanied by a change in the 
physical and chemical properties 
of the matter. Carbon, for 
example, occurs in carbonate 
rocks such as limestone, in coal 
and other fossil fuels, in the 
atmosphere as carbon dioxide 
gas, in water as dissolved carbon 
dioxide, and in all organisms as 
complex molecules that control 
the chemistry of life. 
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Grade 4 
Earth in Space and Time 


Exhibit 12. Earth and Space Sciences content statements for 


rades 4, 8, and 12 
Grade 12 


Objects in the Universe: Patterns in the sky (4), a model of the solar system (8), a vision of the 


universe (12). 


E4.1: Objects in the sky have patterns 
of movement. The Sun, for example, 
appears to move across the sky in the 
same way every day, but its path 
changes slowly over the seasons. The 
Moon appears to move across the sky 
on a daily basis much like the Sun. 


E4.2: The observable shape of the 
Moon changes from day to day ina 
cycle that lasts about a month. 


E8.1: In contrast to an earlier theory 
that Earth is the center of the universe, 
it is now known that the Sun, an 
average star, is the central and largest 
body in the solar system. Earth is the 
third planet from the Sun in a system 
that includes seven other planets and 
their moons, as well as smaller objects 
such as asteroids and comets. 


E8.2: Gravity is the force that keeps 
most objects in the solar system in 
regular and predictable motion. These 
motions explain such phenomena as 
the day, the year, phases of the Moon, 
and eclipses. 


E12.1: The origin of the universe 
remains one of the greatest questions 
in science. The “big bang” theory 
places the origin approximately 13.7 
billion years ago when the universe 
began in a hot, dense state. According 
to this theory, the universe has been 
expanding ever since. 


E12.2: Early in the history of the 
universe, matter (primarily the light 
atoms hydrogen and helium) clumped 
together by gravitational attraction to 
form countless trillions of stars and 
billions of galaxies. 


E12.3: Stars, like the Sun, transform 
matter into energy in nuclear 
reactions. When hydrogen nuclei fuse 
to form helium, a small amount of 
matter is converted to energy. These 
and other processes in stars have led 
to the formation of all the other 
elements. 
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Exhibit 12 (continued). Earth and Space Sciences content statements for grades 4, 8, and 12 
Grade 8 


Grade 4 


Grade 12 


E4.3: The surface of Earth changes. 
Some changes are due to slow 
processes such as erosion and 
weathering, and some changes are due 
to rapid processes such as landslides, 
volcanic eruptions, and earthquakes. 


E8.3: Fossils provide important 
evidence of how life and 
environmental conditions have 
changed in a given location. 


E8.4: Earth processes seen today, 
such as erosion and mountain 
building, make it possible to measure 
geologic time through methods such 
as observing rock sequences and using 
fossils to correlate the sequences at 
various locations. 


E12.4; Early methods of determining 
geologic time, such as the use of 
index fossils and stratigraphic 
sequences, allowed for the relative 
dating of geological events. However, 
absolute dating was impossible until 
the discovery that certain radioactive 
isotopes in rocks have known decay 
rates, making it possible to determine 
how many years ago a given rock 
sample formed. 


E12.5: Theories of planet formation 
and radioactive dating of meteorites 
and lunar samples have led to the 
conclusion that the Sun, Earth, and the 
rest of the solar system formed from a 
nebular cloud of dust and gas 4.6 
billion years ago. 


E12.6: Early Earth was very different 
from today’s planet. Evidence for 
one-celled forms of life (bacteria) 
extends back more than 3.5 billion 
years. The evolution of life caused 
dramatic changes in the composition 
of Earth’s atmosphere, which did not 
originally contain molecular oxygen. 


E12.7: Earth’s current structure has 
been influenced by both sporadic and 
gradual events. Changes caused by 
violent earthquakes and volcanic 
eruptions can be observed on a human 
time scale; however, many geological 
processes, such as the building of 
mountain chains and shifting of entire 
continents, take place over hundreds 
of millions of years. 
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Exhibit 12 (continued). Earth and Space Sciences content statements 


Grade 4 
Earth Structures 


atmosphere (8). 


Grade 8 


Properties of Earth Materials: Natural and manmade materials (4), soil analysis and layers of the 


for grades 4, 8, and 12 


Grade 12 


E4.4: Earth materials that occur in 
nature include rocks, minerals, soils, 
water, and the gases of the 
atmosphere. 


E4.5: Natural materials have different 
properties that sustain plant and 
animal life. 


E4.6: Some Earth materials have 
properties either in their present form 
or after design and modification that 
make them useful in solving human 
problems and enhancing the quality of 
life, as in the case of materials used 
for building or fuels used for heating 
and transportation. 


E8.5: Rocks and rock formations bear 
evidence of the minerals, materials, 
temperature/pressure conditions, and 
forces that created them. Some 
formations show evidence that they 
were deposited by volcanic eruptions. 
Others are composed of sand and 
smaller particles that are buried and 
cemented by dissolved minerals to 
form solid rock again. Still others 
show evidence that they were once 
earlier rock types that were exposed to 
heat and pressure until they changed 
shape and, in some cases, melted and 
recrystallized. 


E8.6: Soil consists of weathered rocks 
and decomposed organic material 
from dead plants, animals, and 
bacteria. Soils are often found in 
layers with each having a different 
chemical composition and texture. 


E8.7: The atmosphere is a mixture of 
nitrogen, oxygen, and trace gases that 
include water vapor. The atmosphere 
has a different physical and chemical 
composition at different elevations. 
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Exhibit 12 (continued). Earth and Space Sciences content statements for grades 4, 8, and 12 


Grade 4 


Earth Structures 


Grade 12 


Tectonics: The basics of tectonic theory and Earth magnetism (8), the physical mechanism that 


drives tectonics and its supporting evidence (12). 


E8.8: Earth is layered with a 
lithosphere; a hot, convecting mantle; 
and a dense, metallic core. 


E8.9: Lithospheric plates on the scale 
of continents and oceans constantly 
move at rates of centimeters per year 
in response to movements in the 
mantle. Major geological events, such 
as earthquakes, volcanic eruptions, 
and mountain building, result from 
these plate motions. 


E8.10: Earth as a whole has a 
magnetic field that is detectable at the 
surface with a compass. Earth’s 
magnetic field is similar to the field of 
a natural or manmade magnet with 
north and south poles and lines of 
force. For thousands of years, people 
have used compasses to aid in 
navigation on land and sea. 


E12.8: Mapping of the Mid-Atlantic 
Ridge, evidence of sea floor 
spreading, and subduction provided 
crucial evidence in support of the 
theory of plate tectonics. The theory 
currently explains plate motion as 
follows: the outward transfer of 
Earth’s internal heat propels the plates 
comprising Earth’s surface across the 
face of the globe. Plates are pushed 
apart where magma rises to form 
midocean ridges, and the edges of 
plates are pulled back down where 
Earth materials sink into the crust at 
deep trenches. 
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Exhibit 12 (continued). Earth and Space Sciences content statements for 


Grade 4 


rades 4, 8, and 12 
Grade 12 


E4.7: The Sun warms the land, air, 
and water and helps plants grow. 


E4.8: Weather changes from day to 
day and during the seasons. 


E4.9: Scientists use tools for 
observing, recording, and predicting 
weather changes from day to day and 
during the seasons. 


E8.11: The Sun is the major source of 
energy for phenomena on Earth’s 
surface. It provides energy for plants 
to grow and drives convection within 
the atmosphere and oceans, producing 
winds, ocean currents, and the water 
cycle. 


E8.12: Seasons result from annual 
variations in the intensity of sunlight 
and length of day, due to the tilt of 
Earth’s rotation axis relative to the 
plane of its yearly orbit around the 
Sun. 


E8.13: Global patterns of atmospheric 
movement influence local weather. 
Oceans have a major effect on climate 
because water in the oceans holds a 
large amount of heat. 


E12.9: Earth systems have internal 
and external sources of energy, both 
of which create heat. The Sun is the 
major external source of energy. Two 
primary sources of internal energy are 
the decay of radioactive isotopes and 
the gravitational energy from Earth’s 
original formation. 


E12.10: Climate is determined by 
energy transfer from the Sun at and 
near Earth’s surface. This energy 
transfer is influenced by dynamic 
processes such as cloud cover, 
atmospheric gases, and Earth’s 
rotation, as well as static conditions 
such as the positions of mountain 
ranges, oceans, seas, and lakes. 
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Exhibit 12 (continued). Earth and Space Sciences content statements for 


Grade 4 


rades 4, 8, and 12 
Grade 12 


Earth Systems 


Biogeochemical Cycles: Uses of Earth resources (4), natural and human-induced changes in Earth 
materials and systems (8), biogeochemical cycles in Earth systems (12). 


E4.10: The supply of many Earth 
resources such as fuels, metals, fresh 
water, and farmland is limited. 
Humans have devised methods for 
extending the use of Earth resources 
through recycling, reuse, and renewal. 


E4.11: Humans depend on their 
natural and constructed environment. 
Humans change environments in ways 
that can either be beneficial or 
detrimental for themselves and other 
organisms. 


E8.14: Water, which covers the 
majority of Earth’s surface, circulates 
through the crust, oceans, and 
atmosphere in what is known as the 
water cycle. Water evaporates from 
Earth’s surface, rises and cools as it 
moves to higher elevations, condenses 
as clouds, falls as rain or snow, and 
collects in lakes, oceans, soil, and 
underground. 


E8.15: Human activities, such as 
reducing the amount of forest cover, 
increasing the amount and variety of 
chemicals released into the 
atmosphere, and intensive farming, 
have changed Earth’s land, oceans, 
and atmosphere. Studies of plant and 
animal populations have shown that 
such activities can reduce the number 
and variety of wild plants and animals 
and sometimes result in the extinction 
of species. 


E12.11: Earth is a system containing 
essentially a fixed amount of each 
stable chemical atom or element. 
Most elements can exist in several 
different chemical forms. Earth 
elements move within and between 
the lithosphere, atmosphere, 
hydrosphere, and biosphere as part of 
biogeochemical cycles. 


E12.12: Movement of matter through 
Earth’s systems is driven by Earth’s 
internal and external sources of 
energy. These movements are often 
accompanied by a change in the 
physical and chemical properties of 
the matter. Carbon, for example, 
occurs in carbonate rocks such as 
limestone, in coal and other fossil 
fuels, in the atmosphere as carbon 
dioxide gas, in water as dissolved 
carbon dioxide, and in all organisms 
as complex molecules that control the 
chemistry of life. 


E12.13: Natural ecosystems provide 
an array of basic processes that affect 
humans. These processes include 
maintenance of the quality of the 
atmosphere, generation of soils, 
control of the hydrologic cycle, 
disposal of wastes, and recycling of 
nutrients. 
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COMPONENTS OF SCIENCE CONTENT AS ASSESSMENT ITEM CONTEXTS 


Science-literate citizens should be familiar with certain components of science content, 
such as the history and nature of science and the relationship between science and tech- 
nology. These features are highly valued by science educators and are viewed as critical 
to the teaching and learning of science (AAAS 1993; NRC 1996). In chapter three, the 
nature of science is partially addressed through a discussion of scientific inquiry. Similar- 
ly, the relationship between science and technology is partially addressed in chapter three 
through a discussion of technological design. In addition, these components of 

science content will be incorporated into the contexts of assessment items; they will not 
be directly assessed because of time and resource constraints. Chapter four contains addi- 
tional information. 


FROM SCIENCE CONTENT TO SCIENCE PRACTICES 


This chapter has presented the science content that defines the NAEP Science Assess- 
ment content domain. The content statements, as presented in this chapter, do not 
describe students’ performances in observable terms. Chapter three contains a description 
of science practices and cognitive demands. It also shows how science content statements 
can be combined (crossed) with science practices to generate performance expectations 
(i.e., descriptions of students’ expected and observable performances on the NAEP 
Science Assessment). Based on these performance expectations, assessment items can be 
developed and then inferences can be derived from student responses about what students 
know and can do in science. Chapter three provides an illustrative example of this 
process. 
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CHAPTER THREE 
SCIENCE PRACTICES 


Chapter two presented content statements that define the key science principles (as well 
as the facts, concepts, laws, and theories) that NAEP will assess. However, NAEP will 
assess not only science content statements but also the ways in which knowledge is used. 
This chapter defines what students should be able to do with the science content state- 
ments by articulating key science practices that NAEP will assess: Identifying Science 
Principles, Using Science Principles, Using Scientific Inquiry, and Using Technological 
Design. These practices are useful for generating science-rich assessment items. 


To help assessment developers, the science practices can be associated with the cognitive 
demands they place on students. This chapter employs a set of four cognitive demands: 
(1) “knowing that,” (2) “knowing how,” (3) “knowing why,” and (4) “knowing when and 
where to apply knowledge.” These cognitive demands help ensure that NAEP assessment 
items are developed in a way that will elicit the kinds of knowledge and thinking that 
underlie the framework’s performance expectations (as explained later in this chapter). 
They also provide a tool for interpreting students’ responses on the assessment items. 


This chapter shows how science content statements can be combined or crossed with 
practices to generate performance expectations, which then guide the development of 
assessment items. By comparing student responses with the particular science content and 
practice being assessed, inferences about what students know (about particular science 
principles) and can do (with respect to particular science practices) are made. 


Two types of textboxes are used throughout this chapter. Clarification textboxes provide 
details on potentially confusing topics, such as the distinction between Identifying 
Science Principles and Using Science Principles. Illustrative item textboxes provide 
assessment items that exemplify recommendations discussed in the text. Answers to 
selected-response items are indicated within the textbox; appendix C contains scoring 
guides for constructed-response items. Although items in these textboxes may assess 
more than one content statement or practice, only the primary content and practice desig- 
nations are provided. This follows NAEP practice, which uses only primary designations 
for items in the analysis and reporting of student responses. 


OVERVIEW OF PRACTICES 


Over the course of human history, people have developed many interconnected and vali- 
dated ideas about the physical and biological world. These ideas have enabled successive 
generations to achieve an increasingly comprehensive and reliable understanding of the 

natural world. Scientific ideas are generated and verified by observing natural phenome- 
na, finding patterns in these observations, and constructing theoretical models to explain 
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these patterns.’ The patterns and models can in turn be used to describe, measure, Classi- 
fy, explain, and predict other observations. Science knowledge is used to reason about the 
natural world and to improve the quality of scientific thought and action. Hence, NAEP 
will assess how well 4th, 8th, and 12th grade students can engage in the following broad- 
ly organized science practices: 


Identifying Science Principles 
Using Science Principles 
Using Scientific Inquiry 
Using Technological Design 


Because these practices are closely related, the categories are not distinct and some over- 
lap is expected. 


The ability to communicate accurately and effectively is essential in science, and this 
expectation is a strand that runs across the practices. Accurate and effective communica- 
tion may include (but is not limited to) writing clear instructions that others can follow to 
carry out an investigation; reading and organizing data in tables and graphs; locating 
information in computer databases; using audio, video, multimedia, and other technolo- 
gies to access, process, and integrate scientific findings; using language and scientific 
terms appropriately; drawing pictures or schematics to aid in descriptions of observa- 
tions; summarizing the results of scientific investigations; and reporting to various 
audiences about facts, explanations, investigations, and data-based alternative explana- 
tions and designs (AAAS 1993; NRC 1996; Partnership for 21st Century Skills 2004). 
Quantitative reasoning is also fundamental to science. It is the capacity not only to calcu- 
late (e.g., determine density given an object’s mass and dimensions) but also to model a 
system (e.g., determine the energy released from a chemical reaction). 


SOURCES FOR THE DEVELOPMENT OF PRACTICES 


The framework developers examined a number of sources to develop the short list of 
practices to be assessed in the NAEP Science Assessment. The most important sources 
were the Science as Inquiry sections of the National Standards and chapter 12: Habits of 
Mind in Benchmarks. The committee also consulted the National Standards and Bench- 
marks sections on Science and Technology and The Designed World, and the Validities 
of Science Inquiry Assessments project (Quellmalz et al. 2005). Conducted by SRI Inter- 
national during 2001-05, this project classified assessment items according to the inquiry 
standards discussed in the National Standards. The practices described below are found 
in most of the above sources. Cognitive research on science learning, international 
frameworks, and state standards were also used as reference points. 


” Because natural phenomena are understood and described based on collected observations, the terms 
“phenomena” and “observations” are intricately intertwined. For ease of communication, the framework 
uses the term “observations” to represent both specific observations of a natural phenomenon and the 
phenomenon itself. 
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IDENTIFYING SCIENCE PRINCIPLES 


This category focuses on students’ ability to recognize, recall, define, relate, and repre- 
sent basic science principles specified in the Physical Science, Life Science, and Earth 
and Space Sciences content statements presented in chapter two. The content statements 
themselves are often closely related to one another conceptually. Moreover, the science 
principles included in the content statements can be represented in a variety of forms, 
such as words, pictures, graphs, tables, formulas, and diagrams (AAAS 1993; NRC 
1996). NAEP will assess students’ ability to describe, measure, or classify observations; 
state or recognize principles included in the content statements; connect closely related 
content statements; and relate different representations of science knowledge. The prac- 
tices assessed in this category draw on declarative knowledge (or “knowing that”), which 
is described in the Cognitive Demands section later in this chapter. Identifying Science 
Principles comprises the following general types of performance expectations: 


e Describe, measure, or classify observations (e.g., describe the position and motion 
of objects; measure temperature; classify relationships between organisms as 
being predator/prey, parasite/host, producer/consumer). 


e State or recognize correct science principles (e.g., mass is conserved when 
substances undergo changes of state; all organisms are composed of cells; the 
atmosphere is a mixture of nitrogen, oxygen, and trace gases that include water 
vapor). 


e Demonstrate relationships among closely related science principles (e.g., connect 
statements of Newton’s three laws of motion, relate energy transfer with the water 
cycle). 


e Demonstrate relationships among different representations of principles (e.g., 
verbal, symbolic, diagrammatic) and data patterns (e.g., tables, equations, graphs). 


Identifying Science Principles is integral to all of the other science practices. 


The following two items illustrate the expectation that students will recognize correct 
science principles. The first item assesses students’ ability to correctly identify simple 
information about the location of bodies within the solar system (declarative knowledge). 
More than half of the eighth graders nationally answered it incorrectly. Thirty-five per- 
cent of the students thought that the Moon is sometimes closer to the Sun than to Earth. 
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Illustrative Item 
The Earth’s Moon is 


. always much closer to the Sun than it is to the Earth. 

. always much closer to the Earth than it is to the Sun. 

. about the same distance from the Sun as it is from the Earth. 

. sometimes closer to the Sun than it is to the Earth and sometimes 
closer to the Earth than it is to the Sun. 


E8.1, Identifying Science Principles 
Source: NAEP 2000, Grade 8. 


Illustrative Item 


Animals and plants are made up of a number of different chemical elements. 
What happens to all of these elements when animals and plants die? 


. They die with the animal or plant. 

. They evaporate into the atmosphere. 

. They are recycled back into the environment. 
. They change into different elements. 


L8.5, Identifying Science Principles 
Source: TIMSS 2003, Grade 8.* 


* TIMSS items appearing in the framework are copyrighted © by the International Association for the 
Evaluation of Educational Achievement. 


USING SCIENCE PRINCIPLES 


Scientific knowledge is useful for making sense of the natural world. Both scientists and 
informed citizens can use patterns in observations and theoretical models to predict and 
explain observations that they make now or that they will make in the future. The practic- 
es assessed in this category draw primarily on schematic knowledge (or “knowing why”) 
in addition to declarative knowledge, which are described in the Cognitive 

Demands section later in this chapter. Using Science Principles comprises the following 
general types of performance expectations: 


e Explain observations of phenomena (using science principles from the content 
statements). 


e Predict observations of phenomena (using science principles from the content 
statements, including quantitative predictions based on science principles that 
specify quantitative relationships among variables). 


NAEP 2015 SCIENCE FRAMEWORK 
64 


e Suggest examples of observations that illustrate a science principle (e.g., identify 
examples where the net force on an object is zero; provide examples of observa- 
tions explained by the movement of tectonic plates; given partial DNA sequences 
of organisms, identify likely sequences of close relatives). 

e Propose, analyze, and/or evaluate alternative explanations or predictions. 


The following item illustrates the expectation that students will predict phenomena. 


Illustrative Item 


Oak tree Caterpillar 


Look at the food web above. If the corn crop failed one year, what 
would most likely happen to the robin population? Explain your 
answer. 


(See appendix C for item scoring guides.) 


L8.6, Using Science Principles 
Source: TIMSS 1999, Grade 8. 


The first two science practice categories—lIdentifying Science Principles and Using 
Science Principles—both require students to correctly state or recognize the science prin- 
ciples contained in the content statements. A difference between the categories is that 
Using Science Principles focuses on what makes science knowledge valuable or, in other 
words, its usefulness in making accurate predictions about phenomena and in explaining 
observations of the natural world in coherent ways (i.e., “knowing why”). Distinguishing 
between these two categories draws attention to differences in the depth and richness of 
individuals’ knowledge of the content statements. Certain actions on the part of students 
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lead to an inference of Identifying Science Principles, while other actions lead to an in- 
ference of Using Science Principles. Assuming a continuum from “just knowing the 
facts” to “using science principles,” there is considerable overlap at the boundary. The 
line between the identifying and using categories is not distinct. Consider the following 
item, which illustrates the expectation that students will connect different representations. 
In this case, the student must identify the correct pictorial representation of a complete 
electrical circuit. 


Illustrative Item 


The pictures show a light bulb connected to a battery. 
Which bulb will light? 


P4.11, Using Science Principles 
Source: TIMSS 2003, Grade 4. 


Student responses to this item are open to two interpretations. If students have had a great 
deal of exposure to these types of circuit representations, their responses would fall under 
Identifying Science Principles. However if, these circuit representations are relatively 
new for students, then they would need to apply more reasoning and their responses 
would fall under Using Science Principles. 
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The following textbox provides further illustration of the distinction between identifying 
the boiling point of water (a fact) and using the relationship between boiling point and 
pressure (altitude) to explain or predict. 


Clarification: Distinguishing Between Identifying Science Principles and Using 
Science Principles—A Boiling Point Example 


Grade 8 Content Statement: Matter—Properties of Matter: 

P8.4: ...Each element and compound has physical and chemical properties, such as 
boiling point, density, color, and conductivity, which are independent of the amount of 
the sample. 


Distinguishing between the two categories of Identifying Science Principles and Using 
Science Principles is a function of actions or performances. Using boiling point as an 
example, one might observe different responses to the question, “What is the boiling 
point of water?” Behaviors or actions might include: 


Penciling in the oval corresponding to 100 ‘C ina selected-response item. 
Writing: “The boiling point of water is 100 ‘C at sea level.” 

Writing: “The boiling point of water changes as pressure changes. So, even 
though water boils at 100 ‘C at sea level (1 atm pressure), it might boil at a lower 
temperature on top of a mountain because pressure is lower up there.” 


The above responses evoke different inferences about the science understanding of the 
individual responding. Both the first and second responses suggest that the question is 
only assessing knowledge of facts or the ability to identify a science principle; however, 
they illustrate the difference between recognizing a correct answer and retrieving that 
correct answer from memory. The third response contains even more sophisticated 
information, suggesting that the student can use a science principle to make predictions. 
Distinctions between these two categories can be clarified by examining student 
responses and conducting cognitive labs. 
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The following item set illustrates a combination of Identifying Science Principles and 
Using Science Principles. It also addresses the expectation that students interpret data 
presented in a graph and use the data to perform a mathematical calculation. 


Illustrative Item 


The graph below shows the distance traveled over time by a student walking 
down a hall. Use the information shown on the graph to do Numbers 7 and 8. 


Distance Traveled Over Time 


Distance (m) 


10 15 20 
Time (seconds) 


7. During which time interval was the student moving the fastest? 


OA 
O B 
Oo 
OD 
Key: D 
P8.14, Identifying Scientific Principles 
8. What was the average speed of the student from 0 seconds to 5 seconds? 


Average speed: 


(See appendix C for item scoring guides.) 


P8.14, Using Scientific Principles 
Source: Colorado Department of Education 2002, Grade 8. 
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USING SCIENTIFIC INQUIRY 


Scientists make observations about the natural world, identify patterns in data, and 
propose explanations to account for the patterns. Although scientists differ greatly from 
one another in the phenomena they study and in how they go about their work, scientific 
inquiry involves the collection of relevant data, the use of logical reasoning, and the 
application of imagination and evidence in devising hypotheses to explain patterns in da- 
ta. Scientific inquiry is a complex and time-intensive process that is iterative rather than 
linear. Scientists are also expected to exhibit, indeed to model, the habits of mind— 
curiosity, openness to new ideas, informed skepticism—that are part of science literacy. 
This includes reading or listening critically to assertions in the media, deciding what evi- 
dence to pay attention to and what to dismiss, and distinguishing careful arguments from 
shoddy ones. These critical thinking and systems thinking skills are the basis for exercis- 
ing sound reasoning, making complex choices, and understanding the interconnections 
among systems (Partnership for 21st Century Skills 2004). Thus, using scientific inquiry 
depends on the practices described above—Identifying Science Principles and Using Sci- 
ence Principles. Moreover, in addition to involving declarative and schematic knowledge, 
Using Scientific Inquiry draws heavily on procedural knowledge (“knowing how,” e.g., 
knowing how to determine the mass of an object). This framework focuses on a few key 
inquiry practices that are practical to measure in the NAEP Science Assessment. Using 
Scientific Inquiry comprises the following general types of performance expectations:® 


e Design or critique aspects of scientific investigations (e.g., involvement of control 
groups, adequacy of sample). 


e Conduct scientific investigations using appropriate tools and techniques (e.g., 
selecting an instrument that measures the desired quantity—length, volume, 
weight, time interval, temperature—with the appropriate level of precision). 


e Identify patterns in data and/or relate patterns in data to theoretical models. 


e Use empirical evidence to validate or criticize conclusions about explanations and 
predictions (e.g., check to see that the premises of the argument are explicit, 
notice when the conclusions do not follow logically from the evidence presented). 


Scientific inquiry is more complex than simply making, summarizing, and explaining 
observations, and it is more flexible than the rigid set of steps often referred to as the 
“scientific method.” The National Standards make it clear that inquiry goes beyond 
“science as a process” to include an understanding of the nature of science (p. 105) and 
further state the following: 


It is part of scientific inquiry to evaluate the results of scientific investiga- 
tions, experiments, observations, theoretical models, and the explanations 


® Tn addition, 12th graders at the Advanced level are expected to be able to identify a scientific question for 
investigation. See appendix B for achievement level descriptions. 


NAEP 2015 SCIENCE FRAMEWORK 
69 


proposed by other scientists. Evaluation includes reviewing the experi- 
mental procedures, examining the evidence, identifying faulty reasoning, 
pointing out statements that go beyond the evidence, and suggesting alter- 
native explanations for the same observations (p. 171). 


In the NAEP Science Assessment, when students use scientific inquiry they are drawing 
on their understanding about the nature of science, including the following ideas (see 
Benchmarks): 


e Arguments are flawed when fact and opinion are intermingled, or the conclusions 
do not follow logically from the evidence presented. 


e A single example can never support the inference that something is always true, 
but sometimes a single example can support the inference that something is not 
always true. 


e If more than one variable changes at the same time in an experiment, the outcome 
of the experiment may not be clearly attributable to any one of the variables. 


e The way in which a sample is drawn affects how well it represents the population 
of interest. The larger the sample, the smaller the error in inference to the popula- 
tion. However, large samples do not necessarily guarantee representation, 
especially in the absence of random sampling. 


NAEP will assess students’ abilities to use scientific inquiry in two ways: students will be 
required to do the practices specified above and students will critique examples of scien- 
tific inquiry. It is incorrect to assume that assessment of using scientific inquiry is best or 
only achieved through hands-on performance tasks and interactive computer tasks 
(ICTs). In both cases of doing and critiquing, some assessment tasks will also be 
presented as paper-and-pencil items. In doing, tasks may present data tables and ask 
students which conclusions are consistent with the data. Other tasks will be presented as 
hands-on performance and/or interactive computer tasks (e.g., students collect data and 
present their results, or students specify experimental conditions on computer simulations 
and observe the outcomes). In regard to critiquing, students might be asked to identify 
flaws in a poorly designed investigation or suggest changes in the design to produce more 
reliable data. Tasks may be based on print or electronic media (e.g., items may ask 
students to suggest alternative interpretations of data described in a newspaper article). 
Chapter four contains more information on types of items. 
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The following middle school (grade 8) item illustrates the expectation that students will 
conduct scientific investigations. By manipulating the simulation, students gather data 
and solve the problems that are presented. 


Illustrative Item 


This interactive computer task is one module in an extended assessment of students’ abilities to 
use a range of technologies to investigate a complex problem, “Should lynx be reintroduced 
into a national park?” Students accessed, organized, and analyzed data on the number of hares 
in the park over a 25-year period, researched factors that would impact the population, and 
created a graph to analyze the trend. (See appendix C for a description of the full task.) 


This module allows students to interact with a simulated predator/prey (lynx/hare) population 
model. Students use the modeling tool to observe population trends that result from different 
parameter values for the lynx and hare populations. The screen shot below is an example of 
what students see after they have selected parameters and run the simulation. Note that it is a 
single screen shot and represents only a small subset of the many screens that students actually 
see when engaged in this interactive computer task. After students have run the modeling 
software, they are asked a series of questions (e.g., size of the hare population over time). 


| Sees 


Initial Number of Hares 


time (years) 


Hare and Lynx Populations 
Many 


Few 


L8.6, Using Scientific Inquiry 
Source: Quellmalz et al. 2004. 
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USING TECHNOLOGICAL DESIGN 


In both the National Standards and Benchmarks, the term “technological design”” refers 


to the process that underlies the development of all technologies, from paper clips to 
space stations. The National Standards explain that this meaning “is not to be confused 
with ‘instructional technology,’ which provides students and teachers with exciting 
tools—such as computers—to conduct inquiry and to understand science” (p. 24). 


In the framework, using technological design describes the systematic process of apply- 
ing science knowledge and skills to solve problems in a real-world context. The National 
Standards clearly state the reason for including technological design in the science 
curriculum: “Although these are science education standards, the relationship between 
science and technology is so close that any presentation of science without developing an 
understanding of technology would portray an inaccurate picture of science” (p. 190). 
The National Standards define technology and its relationship to science as follows: 


As used in the Standards, the central distinguishing characteristic between 
science and technology is a difference in goal: The goal of science is to 
understand the natural world, and the goal of technology is to make modi- 
fications in the world to meet human needs. Technology as design is 
included in the Standards as parallel to science as inquiry (p. 24). 


As in scientific inquiry, the professional practice of technological design (also called 
engineering design) is complex and time intensive. Because NAEP addresses the subject 
area of science, the use of technological design components in the NAEP Science 
Assessment will be limited to those that reveal students’ abilities to apply science princi- 
ples in the context of technological design. Students’ abilities to Identify and Use Science 
Principles should provide the opportunities as well as the limits for assessment tasks 
related to Using Technological Design. For example, if students are asked to design a 
town’s energy plan, they may be expected to consider the environmental effects of using 
natural gas versus using coal, but they would not be expected to consider the economic, 
political, or social ramifications of such a plan. 


The framework samples key components of Using Technological Design from the more 

complete descriptions found in the National Standards and Benchmarks. Using Techno- 

logical Design comprises the following general types of performance expectations, all of 
which involve students using science knowledge to accomplish the following: 


e Propose or critique solutions to problems, given criteria and scientific constraints. 


e Identify scientific tradeoffs in design decisions and choose among alternative 
solutions. 


* This practice is discussed in some detail because it is new in NAEP Science Assessments. 
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e Apply science principles or data to anticipate effects of technological design 
decisions. 


The three components of Using Technological Design are discussed below. 


First, the technological design process is rooted in the definition of a problem that can be 
solved through a technological design process. The problem generally describes a human 
need or want and specifies criteria and constraints for an acceptable solution (Internation- 
al Technology Education Association 2000). Only constraints that reflect the science con- 
tent statements in this framework will be considered in developing relevant NAEP as- 
sessment items. The engineer who designs a bridge, for example, must take into 

account the effects of wind and water currents by using relevant physics principles to 
simulate these effects on possible structures before the bridge is built. 


Second, even if limited to the application of science principles, choosing between alterna- 
tive solutions almost always involves tradeoffs. As stated in the Benchmarks: 


There is no perfect design. Designs that are best in one respect ... may be 
inferior in other ways. ... Usually some features must be sacrificed to get 
others. How such trade-offs are received depends upon which features are 
emphasized and which are down-played (p. 49). 


The application of science principles may be used to compare alternative technological 
solutions to see which will better solve the problem and accomplish the project’s goals. 


Finally, while the chosen solution may be intended to solve a human problem or meet a 
human need, the effects are not always as planned. When the automobile was invented, 
no one could have predicted the environmental and human health impacts of vehicle 
emissions. However, scientists and engineers must work together to apply their 
knowledge of the natural world to make such predictions. According to the National 
Standards, students in grades K—4 should know about the effects of design solutions: 


People continue inventing new ways of doing things, solving problems, 
and getting work done. New ideas and inventions often affect other 
people; sometimes the effects are good and sometimes they are bad. It is 
helpful to try to determine in advance how ideas and inventions will affect 
other people (p. 140). 


In terms of cognitive demands, both declarative (knowing that) knowledge and schematic 
(knowing why) knowledge come into play for the three components of Using Technolog- 
ical Design, as does strategic knowledge (knowing when and where to apply knowledge). 


The role of technological design in U.S. science classrooms currently varies widely, and 
it is not possible to predict the extent to which it will be integrated into the school 
curriculum in the future. The role of technological design in the NAEP Science 
Assessment will need to be revisited regularly in response to its evolving role in school 
science. 
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Since using technological design in the NAEP Science Assessment must have direct rele- 
vance to science, it is assumed that students have some understanding of the relationship 
between science and technology. The science-technology relationship is further discussed 
in chapter four as providing context for assessment items. 


The following item illustrates the expectation that students will apply science principles 
to anticipate the effects of a technological design decision. 
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Illustrative Item 


Occasionally, a fire will destroy a forest, burning down trees and push- 
ing wildlife out of their forest homes. However, the forest will grow 
back. Eventually, through the process of forest succession as shown be- 
low, short grasses and flowers begin to grow and animals make new 
homes. 


Forest Succession 


Short grasses 


grasses 


Over time, shrubs and trees begin to grow. The forest returns to a lush habitat 
for the wildlife listed in the chart below. 


Forest Wildlife 


Ground dwelling Worms, beetles 

Reptiles and amphibians American toads, wood frogs, snakes, Eastern box turtles 

Small animals Squirrels, chipmunks 

Medium to large animals Opossums, raccoons, white-tailed deer, black bears 

Airborne Butterflies, moths, bees, wild turkeys, red-tailed hawks, bald eagles 


A power company owns part of a forest that was destroyed by a fire. The for- 
est could take decades to rebuild on its own. The company’s department of 
environmental studies suggests planting new trees to help the forest rebuild. 


Using the information in the scenario: 
e Explain how planting trees could benefit the natural ecosystem. 
e Explain how planting trees could harm the natural ecosystem. 


(See appendix C for item scoring guides.) 
E8.15, Using Technological Design 
Source: Washington Assessment of Student Learning 2004, Grade 8. 
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SUMMARY OF PRACTICES 


The general performance expectations for each of the four practices are summarized in 
exhibit 13. Dashed lines indicate that the boundaries between these categories are not dis- 
tinct, and some overlap is to be expected. 


Exhibit 13. General performance expectations for science practices 


€ Communicate accurately and effectively > 


Demonstrate 
: Demonstrate ; : 
ge Describe, State or : : relationships 
Identifying ; relationships 
: measure, or recognize among 
Science ; : among closely ,. 
aod classify correct science ; different 
Principles : ame related science : 
observations. __ principles. ek representations 
principles. eee 
of principles. 
Suggest Propose, 
Explain Predict examples of analyze, and/or 
Using Science observations observations observations evaluate 
Principles of phenomena. of phenomena. that illustratea alternative 
science explanations or 
principle. predictions. 
Use empirical 
Conduct Identify evidence to 
Design or scientific patterns in validate or 
Using critique investigations data and/or criticize 
Scientific aspects of using relate patterns conclusions 
Inquiry scientific appropriate in data to about 
investigations. tools and theoretical explanations 
techniques. models. and 
predictions. 
Propose or critique : ia ghgs : 
Pe q Identify scientific Apply science 
. solutions to : . ae 
Using : tradeoffs in design principles or data to 
, problems given oe ee 
Technological ©... decisions and anticipate effects of 
5 criteria and : ; 
Design ae choose among technological design 
scientific f ‘ ae 
; alternative solutions. decisions. 
constraints. 
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Clarification: Sample Performance Expectations 
for a Life Science Content Statement 


The examples below are all related to the following grade 8 Life Science content statement: 

L8.4: Plants are producers—they use the energy from light to make sugar molecules from the 
atoms of carbon dioxide and water. Plants use these sugars along with minerals from the soil to 
form fats, proteins, and carbohydrates. These products can be used immediately, incorporated into 
the plant’s cells as the plant grows, or stored for later use. 


All examples are also related to a specific situation: 


Two different varieties of grass—one better adapted to full sunlight and one better adapted to 
shade—are each grown in sunlight and in shade. 


The results of a controlled experiment along these lines might resemble the following: 


Condition Grass Type A Grass Type B 


Sunlight Better growth* Less good growth* 


Shade Less good growth* Better growth* 


* Several variables could be used to indicate growth: mass or dry mass of plants, thickness of stems, 
number of new sprouts, etc. 


Identifying Science Principles 


1. State from where a plant’s food originates. 
2. Classify the grass plants as producers or consumers. 


The first performance calls for students to repeat information found in the content statement with 
little or no modification. The second performance asks students to use the definition of producers 


given in the content statement to classify or identify the plants. 


Using Science Principles 


1. Predict whether sugar will move up or down the stems of the grass plants and explain 
your prediction. 
2. Explain where the mass of the growing grass originates. 


These performances require students to use principles in the content statement to predict or 
explain specific observations (growing grass in this case). The content statement itself does not 
provide the answers to the questions. 
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Using Scientific Inquiry 


1. Given a data table showing the mass of grass plants of each type grown in the sunlight 
and shade, draw conclusions about which variety of grass is better adapted to each 
condition. 

List other variables that should be controlled in order to feel confident about your 
conclusions. 


The first performance is related to the content statement in that the importance of light for plant 
growth is useful background information for students. However, the performance requires 
interpretation of new information (the data table) that has to do with differences among types of 
plants, while the content statement contains generalizations about all plants. Thus, the 
performance requires students to use the data to develop new knowledge that they did not have 


previously. The second performance is in part an assessment of the students’ understanding of 
experimental design. However, good answers would also require knowledge of this and related 
content statements to identify variables that are relevant to plant growth. 


Using Technological Design 


1. Given experimental results on the growth of different varieties of grass plants under 
sunlight and shade conditions, develop a plan for using different types of grass seed in 
different parts of a partially shaded park. 


This performance requires students to use knowledge of the content statement and the 
experimental results to accomplish a practical goal (in this case, a park with grass growing well in 
areas that receive varying amounts of sunlight). 


PERFORMANCE EXPECTATIONS 


The NAEP Science Assessment will focus on how students bring science content (as 
described in chapter two) to bear as they engage in the practices described in this chapter. 
That is, science practices are not content-free skills; they require knowledge of the Physi- 
cal, Life, and Earth and Space Sciences as well as knowledge about scientific inquiry and 
the nature of science (e.g., drawing conclusions from investigations). 


Performance expectations are derived from the intersection of content statements and 
science practices. If the content statements from the Physical, Life, and Earth and Space 
Sciences are the columns of a table and the practices (Identifying Science Principles, 
Using Science Principles, Using Scientific Inquiry, Using Technological Design) are the 
rows, the cells of the table are inhabited by performance expectations. Examples are pro- 
vided in exhibit 14, which is based on exhibit 1 in chapter one. Note that performance 
expectation cells may overlap because the content and practice categories themselves are 
not distinct (as indicated by dashed lines). 
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Exhibit 14. Generating examples of grade 8 performance expectations 


Identifying 
Science 
Principles 


Science Content 


Physical Science 
Content Statements 


Identify the units that 
might be used to 
measure the speed of 
an ant and the speed of 
an airplane (see 
P8.14).* 


Life Science 
Content Statements 


Identify the raw 
materials that plants 
use to make sugars 
(see L8.4). 


Earth and Space 
Sciences 
Content Statements 

Identify wind as the 
movement of air from 
higher to lower 
pressure regions (see 
E8.11). 


An object (e.g., a toy 
car) moves with a 
constant speed along a 
straight line. Predict 


Explain why sugars are 
found to move 
primarily down the 
stem of a growing 


Explain why mountain 
soils are generally 
thinner than floodplain 
soils (see E8.6). 


goes down a hill (see 
P8.16). 


Using Science (with justification) plant (e.g., potato, 
Principles | what might happen to | carrot) (see L8.4). 
this object’s speed as it 
2 rolls downhill (see 
= P8.16). 
S 
cy Design an experiment | Criticize conclusions Given data (indexed by 
g to determine how the about likely month) on annual 
= speed of a battery- consequences of trends of incoming 
n Usi operated toy car consuming various solar radiation for five 
sing oH 
Scientific changes as a result of diets based on flawed | cities, determine 
Inquiry added mass (see premises or flaws in whether the location is 
P8.16). reasoning (see L8.5). in the Northern or 
Southern Hemisphere 
(see E8.12). 
Evaluate the following | Identify possible Describe the 
car designs to ecological side effects | consequences (e.g., 
: determine which one is | of agricultural erosion) of 
Using ; an a pe : : 
: most likely to maintain | fertilizer runoffintoa | undercutting a steep 
Technological ; 
Design a constant speed as it lake (see L8.7). slope for a road cut 


(see E8.4). 


* To identify the science content statement on which each performance expectation is based, the content 


statement’s unique code (from exhibits 8, 10, and 12 in chapter two) is provided. 
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The content statements from chapter two on which these performance expectations are 
based are written in general terms. The process of creating performance expectations 
requires further clarification of the content statements themselves. As described in 
chapter two, this involves “detailing” the meanings of the content statements and setting 
boundaries on the content to be assessed at a given grade level. Moreover, if the crossing 
of content statements with practices were done for every science content statement and 
practice, the number of performance expectations generated could be unmanageably 
large. Exhibit 15 provides selected example performance expectations for a single Earth 
and space sciences content statement. The specifications include additional examples. 
These examples are illustrative, not exhaustive. It is expected that assessment developers 
will continue this process for all content and practices sampled for a particular NAEP 
Science Assessment. 


Performance expectations are written with particular verbs indicating the desired perfor- 
mance expected of the student. The action verbs associated with each practice are not 
firmly fixed, and the use of any action verb must be contextualized. For example, when 
the science practice component “conduct scientific investigations” is crossed with a 
states-of-matter content statement, it can generate a performance expectation that em- 
ploys a different action verb, such as “heats as a way to evaporate liquids.” 


GENERATING AND INTERPRETING ITEMS 


Neither the content statements from chapter two nor the practices discussed in this chap- 
ter will be assessed in isolation. All assessment items will be derived from a combination 
of the two (i.e., from performance expectations). Observed student responses to these 
items can then be compared with expected student responses to make inferences about 
what students know and can do. Exhibit 15 is an Earth and Space Sciences example of 
the process of generating and interpreting items from performance expectations. The item 
examples are of two types: item suggestions (descriptions of items to be developed) and 
illustrative items (released items from various large-scale assessments). Additional 
examples of the process of generating and interpreting items are provided in the specifi- 
cations. 
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Exhibit 15. Earth and Space Sciences example of generating and interpreting items 
Grade 8: Earth in Space and Time—Objects in the Universe 


Content Statement 


E8.2: Gravity is the force that keeps most objects in the solar system in regular and predictable motion. 
Those motions explain such phenomena as the day, the year, phases of the Moon, and eclipses. 


Commentary 
This content statement encompasses two interrelated sets of concepts: 


1. Gravity acts between and among all objects in the solar system, and it plays an essential role in the regular 
and predictable motions of planets around the Sun and of satellites around planets. 


e On Earth, gravity is experienced as a force that pulls everything “down” toward the center of the 
Earth. (A common naive conception is that the atmosphere “pushes” things down and causes 
gravity.) 

e Gravity is a force of attraction that is exerted by every object on every other object. 

e Gravity exists in space and on other planets. (A common naive conception among students is that 
there is no gravity in space because space has no air.) 

e = The almost circular motion of planets and satellites results from the force of gravity and the 
tendency of a body to continue moving through space in a straight line unless acted upon by a net 
force. 


2. The regular and predictable motions of Earth, the Sun, and the Moon cause the cyclic phenomena that can 
be observed in the sky. 


e The day-night cycle results from Earth’s rotation on its axis once in 24 hours. 

e Annual changes in the visible constellations and the seasons result from Earth’s revolution around 
the Sun once every 365'/, days. 

e Moon phases result from the Moon’s orbit around Earth about once a month, which changes the 
part of the Moon that is lighted by the Sun and how much of the lighted part can be seen from 
Earth. 


Note the connection between this content statement and the Physical Science subtopic, forces affecting 
motion. 


Students are not expected to use the inverse square relationship of gravitational force and distance to find 
the strength of the gravitational force between two objects. 


Students do not need to know that the motion of planets and satellites is elliptical and not circular. 
Examples of Performance Expectations 


Identifying Science Principles. Students can: 
e Identify gravity as the force exerted by every object in the solar system on every other object. 
e Identify gravity as the force that keeps the Moon circling Earth, rather than flying off into space. 
e Describe the regular motions of Earth through space, including its daily rotation on its axis and its 
yearly motion around the Sun. 


NAEP 2015 SCIENCE FRAMEWORK 
81 


Exhibit 15 (continued). Earth and Space Sciences example of generating and 
interpreting items 
Examples of Performance Expectations (cont.) 


Using Science Principles. Students can: 
e Explain that the orbit of one object around another is due to the tendency of an object to move ina 

straight line through space and due to the force of gravity between the two objects. 
Explain how the monthly pattern of Moon phases observed from a point on Earth results from the 
Moon’s orbit around Earth, which changes the part of the Moon that is lighted by the Sun and 
what portion of the lighted part can be seen from Earth. 
Distinguish between explanations for lunar (Moon) phases and lunar eclipses. 
Explain that astronauts and other objects in orbit seem to “float” because they are in free fall, 
under the influence of gravity. 


Using Scientific Inquiry. Students can: 
Arrange a set of photographs of the Moon taken over a month’s time in chronological order and 
explain the order in terms of a model of the Earth-Sun-Moon system. 
Design a plan for observing the Sun over a year’s time to find out how the length of the day is 
related to the rising and setting point of the Sun on the horizon. 
Design a series of observations or measurements to determine why some objects—such as certain 
asteroids or comets—visit the solar system just once, never to return. 


Using Technological Design. Students can: 
Choose among several (qualitative) methods for aiming a rocket so that it reaches the planet Mars 
and give a rationale that shows understanding of orbital motion. 
Use scientific tradeoffs in deciding whether or not to support a plan to observe and predict orbits 
of asteroids that enter the inner solar system. 
Given a scenario in which a person is shipwrecked on an island in the ocean, critique plans to 
create a calendar to keep track of the passage of time. 


Items To Assess Identifying Science Principles 
Illustrative Item 1 
What force keeps the planets in our solar system in orbit around the Sun? 


A. gravitational 
B. magnetic 

C. electrical 

D. nuclear 


Key: A 
Source: Adapted from Massachusetts Department of Education, Massachusetts Comprehensive Assessment 
System (MCAS) 2000, Grade 8.* 


** MCAS materials appearing in the framework have been released to the public by the Massachusetts 
Department of Education and are available at no cost at http://www.doe.mass.edu/mcas/testitems.html. 
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Exhibit 15 (continued). Earth and Space Sciences example of generating and 


interpreting items 
Items To Assess Identifying Science Principles (cont.) 


Illustrative Item 2 


The drawings show a rocket being launched from Earth and returning. 


Position 2 


Position 3 


Position 1 


In which of these positions does gravity act on the rocket? 


position 3 only 
positions 1 and 2 only 
positions 2 and 3 only 
positions 1, 2, and 3 


Source: TIMSS 1999, Grade 8. 
Items To Assess Using Science Principles 


Illustrative Item 
A space station is to be located between the Earth and the Moon at the place where 
the Earth’s gravitational pull is equal to the Moon’s gravitational pull. 


On the diagram below, circle the letter indicating the approximate location of the 
space station. 


Earth 


Explain your answer. 


Source: NAEP 1996, Grade 8. 
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Exhibit 15 (continued). Earth and Space Sciences example of generating and 


interpreting items 
Items To Assess Using Science Principles (cont.) 


Interpretation: The correct answer is C. Since the Moon has one-sixth the amount of gravity as Earth, a 
body that experiences an equal gravitational force from Earth and the Moon should be closer to the Moon. 
Point C is the only point that is closer to the Moon. Note: Point C is about one-twelfth of the way between 
the Moon and Earth; it should be one-sixth of the distance. (See appendix C for the item scoring guide.) 


Item Suggestion 


Is there gravity in space? Which of the following gives the best response to this question? 


No. You can see that astronauts float around weightless in their cabin. 

No. There is no air in space, so gravity cannot exist there. 

Yes. There must be gravity since planets keep circling the Sun. 

Some. The Moon has one-sixth as much gravity as Earth, so we know there is some gravity in 
space. 


Key: C 


Interpretation: The correct answer is C. This question is drawn from a series of studies that show a common 
naive conception—that there is no gravity in space because space has no air. The distractors are drawn 
from student interviews. It is likely that these naive conceptions stem from images that students have seen 
of astronauts floating around in a “weightless” environment while in orbit. This item probes schematic 


knowledge (see Cognitive Demands section later in this chapter). 


Item To Assess Using Scientific Inquiry 


Item Suggestion 


A student is presented with a set of photographs of the Moon taken over a month’s time. The photos 
are not presented in chronological order. The student is asked to arrange them in the order in which 
they were taken and explain the reason for Moon phases. 


Interpretation: This suggestion reflects items used frequently in curricular materials (e.g., Schatz and 
Cooper 1994). Students are asked to find patterns in the data. First, they should be sufficiently familiar with 
the lunar cycle to arrange the photographs in order, either in a line to represent a chronology or in a circle 
to represent a cycle (tapping declarative knowledge and procedural knowledge to a lesser extent—see 
Cognitive Demands section later in this chapter). Then, students should be able to explain Moon phases in 
terms of the Moon circling Earth and the changing angle between the Sun and Moon as observed from 
Earth. This part of the item probes schematic knowledge—see Cognitive Demands section later in this 
chapter. This is a challenging question that many educated adults fail to answer. However, studies show 
that middle school students can learn to do this exercise by observing lunar phases and explaining them 
using a model of the Earth-Sun-Moon system (Barnett and Morran 2002; Kavanagh, Agan, and Sneider 
2005). 
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Exhibit 15 (continued). Earth and Space Sciences example of generating and inter- 
preting items 
Items To Assess Using Technological Design 


Item Suggestion 1 


NASA wants to launch a spacecraft with rockets from Earth so that it will reach and orbit Mars. Which 
of the following statements about this flight is WRONG? 


In the first phase of its flight, the forces acting on the spacecraft are the thrust of the rocket engine, 
gravity, and friction from the Earth’s atmosphere. 

When the rocket engine shuts off, the only force acting on the spacecraft is the force of gravity. 
Once the spacecraft is above the Earth’s atmosphere and the rocket engine is off, it will travel at a 
constant speed since there is no gravity in space. 

If the spacecraft is aimed correctly and has the proper speed, the spacecraft will reach Mars and 
require only engine braking to attain orbit. 


Key: C 


Interpretation: The correct answer is C because there is gravity in space and planning for such a rocket 
flight would need to take into account the gravity from Earth, Mars, and the Sun (declarative knowledge— 
see Cognitive Demands section later in this chapter). This question is drawn from a series of studies that 
show the following naive conceptions about gravity are common among many students at the middle 
school, high school, and even college levels: If a body is moving, there is a force acting on it in the 
direction of motion (Finegold and Gorsky 1991; Gunstone and Watts 1985; Sequeira and Leite 1991); there 
is no gravity in space (Bar et al. 1994; Chandler 1991; Morrison 1999); and gravity cannot act in space 
because there is no air in space (Bar and Zinn 1998). One study showed that, with effective instruction, 
middle school students can overcome these naive conceptions and learn that gravity does, in fact, act in 
space, where it keeps satellites and planets in their orbits (Bar, Sneider, and Martimbeau 1997). 


Item Suggestion 2 


Decisions about whether or not to develop new technologies always concern tradeoffs. For example, 
most scientists today believe that the extinction of dinosaurs and many other species was caused by the 
collision of a large asteroid with Earth 65 million years ago. As a result, there is a proposal to develop 
two new technologies: (1) the detection and tracking of all asteroids large enough to do considerable 
damage if they should strike Earth, and (2) the development of a means to send a spacecraft to meet the 
asteroid in space and change its path. Write a paragraph describing: 


a. whether or not you think it would be possible to develop these technologies based on your 
knowledge of science; and 


some of the scientific tradeoffs that should be considered in deciding whether or not to develop 
these new technologies. 


Interpretation: Look for evidence that the students understand what asteroids are, that scientists have 
observed asteroids, and that observations taken at several points in time allow for the prediction of an 
asteroid’s path. Also look for evidence that students understand that spacecraft can be built, launched, and 
navigated to intercept solar system bodies. (In fact, several spacecraft have already intercepted asteroids 
and comets.) Regarding scientific tradeoffs, look for evidence that students recognize the advantages of the 
proposed technologies (e.g., avoid a catastrophic collision in which billions of people and animals could 
die) as well as possible negative effects (e.g., breakup of the asteroid so there are many collisions rather 
than one, accidents on launch). 
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LEARNING PROGRESSIONS 


A learning progression is a sequence of successively more complex ways of reasoning 
about a set of ideas. For any important set of ideas in science, understanding increases 
over time as students learn more, moving from initially naive knowledge of the natural 
world to increasingly more sophisticated knowledge and conceptual understanding; this 
typically occurs in conjunction with educational experiences in and out of school (NRC 
2001). In other words, the progression from novice learner to competent learner to expert 
begins with the acquisition of relevant experiences, principles, concepts, facts, and skills 
and moves to the accumulation and organization of knowledge in a specific domain and 
finally to expertise after extensive experience and practice (e.g., Ericsson 2002). The 
attention paid to growth of understanding may yield rich information about student 
progress. 


Research has been conducted on students’ learning progressions in some areas of science 
and at some levels of students’ development. It is expected that this research will directly 
inform the development of assessment items at grades 4, 8, and 12. For example, NRC 
has commissioned papers on learning progressions in evolution (Catley, Lehrer, and 
Reiser 2005) and in atomic molecular theory (Smith et al. 2004). Learning progressions 
provide opportunities for assessing specific content in greater depth. 


Several caveats about learning progressions are in order. First, learning progressions are 
not developmentally inevitable but depend on instruction interacting with the student’s 
prior knowledge and construction of new knowledge. Thus, learning progressions need to 
invoke assumptions about instruction. Second, there is no single “correct order”; there 
may be multiple pathways by which certain understandings can be reached. The particu- 
lar pathway taken may be influenced by prior instructional experiences, individual 
differences, and current instruction (NRC 1999c, 2001). Thus, learning progressions will 
necessarily be complex, involving multiple specific paths at the micro level, and will 
need to be described in ways that encompass such diversity. Third, actual learning is 
more like ecological succession with changes taking place simultaneously in multiple 
interconnected ways. Thus, attempts to describe specific sequences of learning perfor- 
mance (including those in the Catley, Lehrer, and Reiser (2005) and Smith et al. (2004) 
papers) must inevitably be artificially constrained and ordered. Finally, the learning 
progressions suggested in the framework and specifications are partly hypothetical or 
inferential because long-term longitudinal accounts of learning by individual students do 
not exist. 


Exhibit 16 uses a set of related science content statements across grades 4, 8, and 12 and 
follows the format of the science practices defined in the framework. It illustrates how 
relevant research (e.g., Smith et al. 2004) can be used as an opportunity to assess content 
in greater depth—available research on student learning is used to inform the generation 
of related performance expectations across grades. The exhibit includes examples of 
performance expectations for a possible learning progression for states of matter. These 
illustrative performance expectations do not denote a sense of content priority or im- 
portance and are not a complete representation of the research currently available. 


NAEP 2015 SCIENCE FRAMEWORK 
86 


Exhibit 16. Examples of performance expectations for states of matter 


Grade 4 
(See content statement P4.3.) 


Grade 8 
(See content statement P8.1.) 


Grade 12 
(See content statement P12.1.) 


Identifying Science Principles 


Classify samples of material as 
solid, liquid, or gas. 


Identifying Science Principles 


Given an animation of molecules 
in motion, identify the substance 
that is being illustrated as a solid, 
liquid, or gas. 


Identifying Science Principles 


Explain why ice is harder than 
liquid water in terms of the 
strength of the force between the 
molecules. 


Using Science Principles 


Infer that a change of state (e.g., 
freezing or melting) affects the 
identity of an object but not the 
identity of the material of which 
it is made. 


Using Science Principles 


Predict how the mass of a sample 
of iodine will change after 
sublimation. Justify the prediction 
based on what occurs during 
sublimation at a molecular level. 


Using Science Principles 


Use the concept of molecular 
arrangements and bonds to 
explain why graphite is very soft 
and diamond is very hard, even 
though they are both made of 
pure carbon. 


Using Scientific Inquiry 


Collect, display, and interpret 
data showing how the 
temperature of a substance 
changes over time as it cools and 
becomes a solid. 


Using Scientific Inquiry 


Plan and conduct an investigation 
to determine the melting point 
and boiling point of an unknown 
substance. 


Using Scientific Inquiry 


Using molecular theory, explain 
the results of experiments 
showing how the volume of three 
different liquids changes when 
they are heated. 


Using Technological Design 


Propose a method for determining 
for certain whether holiday 
chocolates that have been shaped 
by different processes (melting, 
freezing, reshaping, or breaking 
into pieces) have the same 
amount of chocolate in them. 


COGNITIVE DEMANDS 


Using Technological Design 


Choose the best solution for 
increasing the altitude of a hot air 
balloon based on an under- 
standing of the macroscopic and 
microscopic changes that occur 
when the gas inside the balloon is 
heated. 


Using Technological Design 


Design an instrument to measure 
temperature as accurately as 
possible, taking into account both 
the thermal properties of liquids 
and solids to be used in the 
device and the structural shape 
and dimensions of the device. 


The four science practices (Identifying Science Principles, Using Science Principles, 
Using Scientific Inquiry, and Using Technological Design) articulate what students 
should know and be able to do with the science principles presented in chapter two. 
Certain ways of knowing and reasoning (cognitive demands) underpin these four science 
practices. In this section, the four cognitive demands—“knowing that,” “knowing how,” 
“knowing why,” and “knowing when and where to apply knowledge”—are discussed 
briefly (see the Specifications for more information). The goal is to further clarify the 
descriptions of the science practices, to facilitate item specifications and item writing, and 
to provide a framework for interpreting students’ responses. That is, the set of four cogni- 
tive demands can be used as a lens to facilitate item development and to analyze student 
responses (Li and Shavelson 2001; Shavelson 2006; Shavelson, Ruiz-Primo, and Wiley 
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2005), thereby checking expectations regarding the science content and practice(s) that 
are being tapped by a given assessment item. '° 


“Knowing that” refers to declarative knowledge. This cognitive demand sets up the 
expectation that students should know and reason with basic science facts, concepts, and 
principles (e.g., density is mass per unit volume) and that they should be able to recall, 
define, represent, use, and relate these basic principles as appropriate. This cognitive 
demand corresponds most closely to the science practice of Identifying Science Princi- 
ples. 


“Knowing how” refers to procedural knowledge. This cognitive demand sets up the 
expectation that students can apply the science facts, concepts, and principles in doing 
science. For example, students should know how to perform simple (routine) and com- 
plex procedures such as systematically observing and recording which objects sink and 
float in water, using a balance scale, measuring an object’s mass, calculating an object’s 
density, and designing and interpreting the results of an investigation (e.g., manipulating 
one variable and holding others constant). Procedural knowledge underlies much of the 
science practice of Using Scientific Inquiry as defined in this framework. 


“Knowing why” refers to schematic knowledge. This cognitive demand sets up the 
expectation that students can explain and predict natural phenomena as well as account 
for how and why scientific claims are evaluated, argued and justified, or warranted (ex- 
plaining and reasoning with principles and models). That is, this cognitive demand deals 
with students’ understanding of how the natural world works (such as why some things 
sink and others float in water), why light is essential to the propagation of most plants, or 
why the Moon changes phases. This cognitive demand overlaps considerably with the 
science understanding expected in the practice of Using Science Principles and also in the 
practices of Using Scientific Inquiry and Using Technological Design. 


The last cognitive demand, “knowing when and where to apply knowledge,” or strategic 
knowledge, is commonly talked about as transfer of current knowledge to new situations 
(tasks or problems). Strategic knowledge involves knowing when and where to use 
science knowledge in a new situation and reasoning through the novel task to reach a 
goal. Strategic knowledge sets up the expectation that students can take their current 
knowledge and apply it to a somewhat novel situation. Such adaptation of knowledge to a 
particular problem and context underlies especially the practices of Using Scientific 
Inquiry and Using Technological Design. 


The cognitive demands are related, not independent (similar to the science practices). 
That is, when explaining “why,” a student will need to call on “knowing that”; at times, 
in justifying “why,” a student may have to call on “knowing how.” In addition, depend- 
ing on the novelty of the task, strategic knowledge (knowing when and where to apply 


'© More than one cognitive demand can be associated with the more complex science practices. These asso- 
ciations may shift according to the knowledge that students at different grade levels bring to an assessment 
task. 
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knowledge) may be called into play. Nevertheless, these related cognitive demands can 
be distinguished, and it is helpful to do so for item development and interpretation of 
student responses. 
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CHAPTER FOUR 
OVERVIEW OF THE ASSESSMENT DESIGN 


This chapter provides an overview of the Specifications. It begins with a brief description 
of the NAEP Science Assessment and a discussion of how items can be set in certain con- 
texts (e.g., history and nature of science) to illustrate components of science content that 
are not otherwise incorporated in the content statements. The types of items to be 
included in the assessment are described and examples are provided in illustrative item 
textboxes. Answers to selected-response items are indicated in the textbox; scoring 
guides for constructed-response items are provided in appendix C. Although items in the 
textboxes may assess more than one content statement or practice, only the primary con- 
tent and practice designations are provided. This follows NAEP practice, which uses only 
primary designations for items in the analysis and reporting of student responses. To 
capture the wide range of science content statements and practices, the assessment will 
contain an array of item types. Consideration is given to English language learners and 
students with disabilities. The chapter concludes with recommendations for small-scale 
special studies. 


OVERVIEW OF THE SCIENCE ASSESSMENT 


The NAEP Science Assessment will include items sampled from the domain of science 
achievement identified by the intersection of the content areas and science practices (i.e., 
performance expectations) at grades 4, 8, and 12. The assessment will include selected- 
response (multiple-choice) items and constructed-response items (which include short 
and extended constructed-response items as well as concept-mapping tasks). Some com- 
bination items may require more than one response; they include item clusters, POE item 
sets, hands-on performance tasks, and interactive computer tasks. The responses 
requested may be all selected-response, all constructed-response, or a mixture. At each of 
grades 4, 8, and 12, student assessment time will be divided evenly between selected- 
response items and constructed-response items. Extra assessment time will be provided 
for a portion of the student sample so that hands-on performance tasks and interactive 
computer tasks can be administered. 


At grade 4, the items will be distributed approximately evenly among Physical Science, 
Life Science, and Earth and Space Sciences. At grade 8, there is a somewhat greater 
emphasis on Earth and Space Sciences. At grade 12, the balance shifts toward Physical 
Science and Life Science, with less emphasis on Earth and Space Sciences. 


Finally, the distribution of items across the science practices will be approximately 60 
percent combining Identifying Science Principles and Using Science Principles, 30 
percent Using Scientific Inquiry, and 10 percent Using Technological Design. From 
grade 4 to 8 to 12, the emphasis on Using Science Principles increases and the emphasis 
on Identifying Science Principles decreases. As students progress through the grades, it is 
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expected that their critical response skills and methodological and analytical capabilities 
will increase. 


ASSESSMENT ITEM CONTEXTS 


Science-literate citizens should be familiar with certain components of science content, 
such as the history and nature of science and the relationship between science and tech- 
nology. Chapter three addresses the nature of science largely through a discussion of the 
science practices (particularly Using Science Principles and Using Scientific Inquiry). 
The relationship between science and technology is partially addressed in a discussion of 
the Using Technological Design practice. The history and nature of science not only 
clarify facets of science practices but also the human aspect of science and the role 
science has played in various cultures. Students can see that science changes, and new 
conclusions can be reached based on new empirical data (e.g., development of the theory 
of plate tectonics, use of gaps in early versions of the Periodic Table to “discover” new 
elements). The reciprocal relationship between science and technology can be seen, for 
example, in that scientists use technological tools to empirically test proposed explana- 
tions for questions about the natural world and engineers develop adaptations to the 
natural world to address human problems, needs, and aspirations based in part on science. 


When items are written to particular content statements, they may be framed in these con- 
textual components of science content. Aspects of the history and nature of science and 
the relationship between science and technology should thus be incorporated into the con- 
texts of assessment items as shown below. 


Illustrative Item 


Emest Rutherford found that when he fired alpha particles at a thin gold foil, some 
were scattered at large angles.* What caused this scattering? 


. The gold’s positive atomic nuclei attracted the negatively charged alpha particles. 
. The gold’s negative atomic nuclei repelled the negatively charged alpha particles. 


. The gold’s negative atomic nuclei attracted the positively charged alpha particles. 
. The gold’s positive atomic nuclei repelled the positively charged alpha particles. 


Key: D 
P12.2, Identifying Science Principles 


* The framework’s developers created this item for illustrative purposes; it requires further development in 
that it is not field tested or published. 


TYPES OF ITEMS 


The judicious selection of items lies at the heart of any effective assessment of science 
achievement. The framework for the 1996-2005 NAEP Science Assessments called for 
multiple-choice items (selected response), open-ended paper-and-pencil items (construct- 
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ed response), and performance exercises. Multiple-choice items made up about 40 per- 
cent of the assessment, as measured by student response time, and open-response items 
made up about 60 percent of assessment time. In addition, subsets of the students sam- 
pled were given an extra 20 minutes in grade 4 and 30 minutes in grades 8 and 12 to 
complete hands-on performance tasks. This framework generally follows the 1996-2005 
recommendations in item structure but also specifies additional item types—some select- 
ed response and others constructed response. As noted above, the item distribution by 
item format in this framework, in terms of student response time, is 50 percent selected- 
response and 50 percent constructed response. 


Two additional considerations are the need to develop items that probe students’ ability 
to use communication skills and quantitative reasoning skills in science (see p. 64). 
Although this framework does not prescribe the amount of assessment time to be spent on 
items that require specific forms of communication or application of mathematics, it is 
expected that items requiring these skills will be represented at all three grade levels. 
Further details will be provided in the specifications. 


JUSTIFICATION FOR VARIATION IN TYPES OF ITEMS 


Issues of time and cost are paramount in any assessment. Accordingly, most of the item 
formats on the NAEP Science Assessment will be traditional selected response and short 
constructed response. However, some more complex items (e.g., hands-on performance 
tasks) should be part of any science assessment. 


Responses to complex items often correlate positively and cluster with responses to sim- 
pler items. However, complex items are recommended in the framework for the follow- 
ing reasons: 


e Items may correlate positively with one another, but they do not necessarily 
measure the same thing; that is, positive correlations can arise even when the cog- 
nitive demands of the assessment items vary. Research has shown that items vary 
in their cognitive demands for different kinds of knowledge and reasoning (e.g., 
Leighton 2004). 


e The NAEP Science Assessment signals the kinds of responses to tasks, problems, 
and exercises, along with the kinds of knowledge and reasoning, that should be 
expected of students as a result of what is taught in the science curriculum (con- 
sistent with the National Standards and Benchmarks). 


For these reasons, the framework calls for a variety of item types for the NAEP Science 
Assessment (see the specifications for further details and explanation). 


DEFINITIONS OF TYPES OF ITEMS 


The framework distinguishes selected-response from constructed-response item formats. 
For selected-response formats, students respond to a question by selecting the answer 


NAEP 2015 SCIENCE FRAMEWORK 
93 


they believe to be most scientifically justifiable from a given set of alternatives. In con- 
trast, with constructed-response formats, students respond to a question by “generating” 
or “constructing” a response. The constructed-response might be a single word, a short 
answer, an essay explanation, a summary of a laboratory investigation using concrete 
materials, or typed responses to a computer simulation. 


In addition to these two main item formats, there are combination items that generally 
require more than one response. These include item clusters, POE item sets, hands-on 
performance tasks, and interactive computer tasks. These combination items can use an 
all-selected-response format, an all-constructed-response format, or a mixture of these 
two main item formats. 


Following are the main types of items to be used on the NAEP Science Assessment: 


1. Selected response 
— Individual multiple-choice items 
2. Constructed response 
— Short constructed-response items 
— Extended constructed-response items 
— Concept-mapping tasks 
3. Combination 
— Item clusters 
— POE item sets 
— Hands-on performance tasks 
— Interactive computer tasks 


Item clusters and POE item sets may use selected-response items, constructed-response 
items, or both. For example, a set of POE items might include a multiple-choice item in 
which students select a prediction and a short constructed-response item in which stu- 
dents are asked to write a justification for their prediction. 


Hands-on performance tasks and interactive computer tasks also may use selected- 
response items, constructed-response items, or both. In recording their answers, students 
may be asked to respond to both selected-response and constructed-response items. For 
example, 12th graders might be asked to manipulate a computer simulation of a chemical 
reaction, a multiple-choice question could ask students to choose the correct mass of a 
reaction product, and a short constructed-response question could ask students to describe 
how mass is conserved in chemical reactions. 


SELECTED-RESPONSE ITEMS 


Selected-response items include individual multiple-choice items. These types of items 
are described below. 
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Individual Multiple-Choice Items 


Selected-response items most often are in a multiple-choice format. Students read, reflect, 
and then select an answer from, for example, four alternatives provided. The alternatives 
include the most scientifically justifiable response (the answer) as well as three “distrac- 
tors.” The distractors should appear plausible to students but should not be scientifically 
justifiable; when feasible, the distractors should also draw from current understanding 
about students’ mental models and learning progressions. Whenever possible and espe- 
cially when the focus is on using science principles or “knowing why” (schematic 
knowledge), naive conceptions, explanations, and predictions of the natural or manmade 
world should serve as distractors. 


Of the following two multiple-choice items, both require identifying science principles 
and both tap the cognitive demand “knowing that” (declarative knowledge). The first 
item taps simple factual content. The second item taps more conceptually sophisticated 
content. For some students, if they cannot easily recall the science content, the second 
item may require the practice of using science principles (i.e., tapping more of “knowing 
why” than “knowing that”). 


Illustrative Item 


Air is made up of many gases. Which gas is found in the greatest amount? 


. Nitrogen 

. Oxygen 

. Carbon dioxide 
. Hydrogen 


E8.7, Identifying Science Principles 
Source: TIMSS 1995, Grade 8. 


NAEP 2015 SCIENCE FRAMEWORK 
95 


Illustrative Item 


The diagram above shows a map of the world with the lines of 
latitude marked. Which of the following places marked on the map is 
most likely to have an average yearly temperature similar to location & ? 


E8.12, Identifying Science Principles 
Source: TIMSS 2003, Grade 8. 


CONSTRUCTED-RESPONSE ITEMS 


Constructed-response items include short constructed-response items, extended con- 
structed-response items, and concept-mapping tasks. 


Short Constructed-Response Items 


This item type generally requires students to supply the correct word, phrase, or quantita- 
tive relationship in response to the question provided in the item; illustrate with a brief 
example; or write a concise explanation for a given situation or result. Thus, students 
must generate the relevant information rather than simply recognize the correct answer 
from a set of given alternatives, as in selected-response items. Following is an example of 
a short constructed-response item. 
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Illustrative Item 


On a hot, humid day the air contains a lot of water vapor. What happens to the water 
vapor in the air when the air becomes very cold? 


(See appendix C for item scoring guides.) 


P4.6, Identifying Science Principles 
Source: TIMSS 2003, Grade 4. 


Extended Constructed-Response Items 


This item type is generally multidimensional; that is, it taps into multiple content state- 
ments, practices, and/or cognitive demands. These types of items can provide useful 
insight into students’ levels of conceptual understanding and reasoning. They can also be 
used to probe students’ ability to communicate in the sciences. Such items generally 
present a situation within or across content areas and require students to analyze the 
situation, choose and carry out an alternative plan for addressing it, and interpret their 
response in light of the original situation. Students may also be given an opportunity to 
explain their responses, reasoning processes, or approach to a problem situation. Howev- 
er, care must be taken (especially with 4th graders and English language learners) to en- 
sure that language ability is not confused with science ability. 


The following item involves reasoning with mental models (on carbon cycling) and thus 
attempts to probe the practice of Using Science Principles and taps into the cognitive 
demand of “knowing why.” 


Illustrative Item 


The biosphere (living organisms), the lithosphere (rocks and soils of Earth’s crust), and 
the atmosphere are all involved in the cycling of carbon atoms. Describe the role that 


each plays in the carbon cycle. 


(See appendix C for item scoring guides.) 
E12.12, Using Science Principles 
Source: New Standards Spring Field Test 1999 for High School.* 
* The New Standards Spring Field Test 1999 for High School is the property of the University of 
Pittsburgh and the National Council on Education and the Economy (NCEE) and may not be used, 
reproduced, or distributed without the express written permission of the University of Pittsburgh and 
NCEE. 
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Concept-Mapping Tasks 


Concept-mapping tasks may be considered a complex item type because of the cognitive 
demands placed on students. Concept maps can be used as a reliable and valid assessment 
of students’ ability to make connections among science principles (Ruiz-Primo and 
Shavelson 1996a). Thus, concept-mapping tasks tap a science ability that is difficult to 
measure by other means. These tasks address the practice of Identifying Science Princi- 
ples and the cognitive demand of “declarative knowledge,” in particular the organization 
of this knowledge. 


In a concept-mapping task, students should be given a set of six to eight concept terms 
and be asked to construct a map linking pairs of terms with directed arrows. Students 
should label each arrow with a word or phrase that explains the relationship between a 
pair of concept terms. An arrow-linked pair of concept terms is called a proposition. 
Students’ concept maps can be evaluated as to the accuracy of propositions in their maps. 
The following textbox provides an illustrative set of instructions for a concept-mapping 
task and a sample student response. See the Specifications for information about scoring 
concept maps. 
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Illustrative Item 


Once students are familiar with how to construct concept maps, they might encounter a set of 
task instructions that resemble the following (adapted from Ruiz-Primo et al. 2001, p. 107): 


Examine the concept terms listed below. The terms selected focus on the topic [insert topic]. 
Construct a concept map using the terms provided below. Organize the terms in relation to one 
another in any way you want. Draw an arrow between the terms you think are related. Label the 
arrow using phrases or only one or two linking words. 

You can construct your map on the blank pages attached. When you finish your map, check that 
(1) all the lines have an arrow, (2) all the arrows have labels, (3) your concept map uses all the 


terms provided, and (4) your map shows what you know about [insert topic]. 


List of terms: [insert list of terms] 


An 11-year-old student constructed the following concept map using terms associated with 
the water cycle: 


is falling 
ee Ee 
water 


comes from 
contain 


goes 
into rivers 


flow to 


MA 


oceans }<— shines 


on 


E8.11, Identifying Science Principles 
Source: White and Gunstone 1992, p. 16. 
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COMBINATION ITEMS 


Combination items include item clusters, POE item sets, hands-on performance tasks, and 
interactive computer tasks. Combination items may consist of an all selected-response 
format, an all constructed-response format, or a mixture of both formats. 


Item Clusters 


The NAEP Science Assessment should include item clusters, and their development 
should be guided by current research on different forms of these items. In this type of 
item set, two or more items focus on an important idea or “mental model.” Hence, these 
items tap the practice of Using Science Principles and the cognitive demand of “knowing 
why.” Where a rigorous body of research is available on students’ conceptions (as there 
is about the solar system), item clusters provide opportunities to assess students’ under- 
standing of a particular key science principle at some depth. This type of item set can 
probe the conceptions and mental models that underlie students’ explanations of and 
reasoning about the natural world. For example, the following questions were part of a set 
of cluster selected-response items probing high school students’ mental models in astron- 
omy (percentages of student responses to each option are given in parentheses): 


Illustrative Items 


What causes day and night? 


. The earth spins on its axis. (66%) 
The earth moves around the Sun. (26%) 

. Clouds block out the Sun’s light. (0%) 

. The earth moves into and out of the Sun’s shadow. (3%) 
The Sun goes around the earth. (4%) 


Key: A 
E8.2, Using Science Principles 


The main reason for its being hotter in summer than in winter is: 


. The earth’s distance from the Sun changes. (45%) 

The Sun is higher in the sky. (12%) 

The distance between the northern hemisphere and the Sun changes. (36%) 
. Ocean currents carry warm water north. (3%) 

An increase occurs in “greenhouse” gases. (3%) 


E8.12, Using Science Principles 
Source: Sadler 1998, pp. 274-276. 


Another approach to probing students’ conceptions of the natural world is to develop a 
cluster of ordered multiple-choice items. These items track students’ performance along a 
learning progression from naive understanding through more reasoned naive conceptions 
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to full and scientifically justified understanding. In this approach, the progression is first 
described and then divided into levels so that multiple-choice items can be designed 
specifically to assess the performance level that a student (or group of students) has 
reached. As noted earlier, constructed-response formats are not precluded from use in 
item clusters. 


POE Item Sets 


POE item sets ask the student to predict, observe, and/or explain as follows: A situation is 
described; the student’s task is to provide a prediction for what will happen (sometimes 
with justification) and/or to provide an explanation for what appears to be an anomaly. 
POE items tend to tap the practice of Using Science Principles and the cognitive demand 
of “knowing why” (schematic knowledge). For example, the following POE item was 
used with middle school students and focuses on prediction based on a mental model of 
buoyancy: 


Illustrative Item 
Rich cut block A into two unequal parts.* Part B is 2/3 of the original block A, and part C 


is 1/3 of the original block A. Block A sinks in water. What will happen to B and C when 
placed in water? 


a | ‘i 


A. Both B and C will float. 
B. B will sink; C will float. 
C. B will subsurface float; C will float. 
D. Both B and C will sink. 


Key: D 
P8.4, Using Science Principles 
Source: Adapted from Shavelson 2006, and Shavelson, Ruiz-Primo, and Wiley 2005. 
* This item assumes that the entire block consists of a completely homogeneous material. The possibility 
that the block is made of heterogeneous material is unlikely to occur to middle school students. 


As noted earlier, a POE item can take either a selected-response or a constructed- 
response format. In the selected-response format, students choose from a set of possible 
alternatives (based on known alternative mental models). In the constructed-response 
format, the student’s task is to write out (with justification) a prediction or an 
explanation. In the POE selected-response item above, students are asked to choose 
among a Set of predictions for whether the parts of the block will sink or float. They can 
choose an answer from several alternatives that include naive conceptions. This item 
could be extended to further probe students’ mental models of buoyancy. Students could 
be asked to justify their answers by selecting from a number of possible explanations; 
again, these alternatives could include the correct explanation along with distractor 
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explanations based on naive conceptions. Or, instead of using a multiple-choice format, 
students could be told that the full block sinks in water and be asked to write their predic- 
tion (with justification) as to what will happen when the two parts are placed in water. Or, 
students could observe a simulation or video of what happens to the full block and the 
two parts and then be asked to explain what they have observed. 


Hands-On Performance Tasks 


Using hands-on performance tasks, students manipulate selected physical objects and try 
to solve a scientific problem involving the objects. These exercises, if carefully designed, 
can probe students’ abilities to combine their science knowledge with the investigative 
skills that reflect the nature of science and inquiry. In large-scale assessments such as 
NAEP, uniform administration must be ensured. In the past, this has been accomplished 
through the use of standardized performance assessment kits, with each exercise proc- 
tored and scored by trained personnel. Special accommodations may be necessary for 
some students. 


A particularly cogent criticism of most hands-on performance tasks administered in 
large-scale assessments is that, rather than tap into students’ ability to inquire into a prob- 
lem, typical performance assessments instead measure students’ ability to follow step-by- 
step instructions to arrive at the expected answer. Assessment developers are likely to 
create these “recipe” types of exercises because they must take into account the vast dif- 
ferences in students’ participation in science courses and their experiences. Given these 
differences, the absence of structure might produce unanticipated responses that could be 
problematic for the assessment either at the time the data are collected or when students’ 
performances are scored by raters. Although both the concern for structure to make large- 
scale assessment manageable and the criticism of highly structured performance tasks are 
well taken, there is evidence that valid performance exercises can be designed, devel- 
oped, administered, and scored without encountering major problems (e.g., Ruiz-Primo 
and Shavelson 1996b; Shavelson, Baxter, and Pine 1991). 


In designing hands-on performance tasks, it is important to keep the following considera- 
tions in mind: The degree to which students engage in some aspect of scientific inquiry 
depends on who selects the problem to be studied, who selects the procedures to be 
carried out in tackling the problem, and who selects the answer. The NAEP assessment 
should provide students with a challenging problem. However, students must have the 
opportunity to determine scientifically justifiable procedures for addressing the problem 
and arriving at a solution. Indeed, the problem to be solved involves setting forth proce- 
dures that manipulate the variable of interest, control extraneous variables, and provide 
solid data to be used in arguing for and justifying a problem solution. In addition to 
allowing students to determine the procedures for carrying out the experiment, NAEP 
hands-on performance tasks should be “content rich” in that they require knowledge of 
science principles to carry them out. 


In brief, any hands-on performance task included in the NAEP assessment should present 
students with a concrete, well-contextualized task (problem, challenge); laboratory 
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equipment and materials; and a response format that leaves the exercise process open. 
Students’ scores should be based on both the procedures created for carrying out the 
investigation and the solution (Shavelson, Baxter, and Pine 1991). The assessment, then, 
should provide the problem that draws on science principles and practices (performance 
expectation of interest) and leave students free to design and carry out the exercise to 
arrive at an answer or solution. The following item, designed for fifth graders, is an 
example of such a task. 


Illustrative Item 


Students are asked to identify the contents of each of the six boxes (A-F) by using the batteries, 
bulbs, and wires they are given to complete a circuit. This task requires knowledge of series 
circuits but leaves problem-solving procedures up to the student. (See appendix C for more 
information about this task.) 


cs! 
ioc ff 


P4.11, Using Scientific Inquiry 
Source: Shavelson, Baxter, and Pine 1991. 


Interactive Computer Tasks 


The NAEP Science Assessment should include some but not necessarily all of the follow- 
ing four types of ICTs: (1) information search and analysis, (2) empirical investigation, 
(3) simulation, and (4) concept mapping. This framework uses static screen shots to illus- 
trate examples of ICTs; however, the screen shots represent only a small subset of the 
many screens students see when engaged in actual ICTs. 


Information search and analysis items pose a scientific problem and ask students to query 
an information database to bring conceptual and empirical information to bear (through 
analysis) on the problem. Following is a screen shot from such an ICT developed for 
eighth graders. 
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Illustrative Item 


39 MINUTES ae ht eee ee ee QUESTION 1 OF 1 $3 


Some scientists study Bock Forword a Add Book mark 


space with large helium 


gas balloons. These 7 
launched from the ground |) NAEP TRE Search Page 


into space but can also be 
launched from spacecraft 
near other planets 


Why do scientists use Enter your search below: 
these gas balloons to Tips for searching 


explore outer space and [gas balloons Go| 
the atmosphere instead of 
using satellites, rockets, or If you want to narrow your search, try placing the word 


other tools? Be sure to "and" between your search terms, 
explain at least three 


advantages of using gas 
balloons 


Base your answer on more Documents 1 to 15 of 300 matching the query “gas balloons". 
than one web page or site 

Be sure to write your 1. Gas Balloons - Westward Gallery 

answer in your own words! 545 


Abstract: Gas Balloons. Frorn Our Exclusive Collection. Gas Balloons 97 - 1. Gas 
You yall be scored on Balloons 97 - 2. Gas Balloons 97 - 3. Gas Balloons 97 - 4. Gas Balloons 97 - 5, 
Gas Balloons 95. Copyright © 1995-2000 by Westward Connections Inc. and/or 
« how well you search, respective content providers. Trade names and trademarks are recognized as 
e the quality of your properties of the 
bookmarks, and 
e how well you answer 
the questions 
om 
Copy = Ctrl+c 
Paste = Ctri+v 
Find = Ctrl +f 


(See appendix C for more information about this task.) 


P8.16, Using Scientific Inquiry 
Source: Persky et al. 2005. 


Empirical investigation items put hands-on performance tasks on the computer and invite 
students to design and conduct a study to draw inferences and conclusions about a prob- 
lem. Whether the computer-simulated experiment assesses the same skills, knowledge, 
and understanding as a hands-on performance task has not been established; a special 
study will be conducted to address this question (see p. 117). Following is a screen shot 
from a computer version of the electric mysteries task (see p. 107 and appendix C for 
more information about this hands-on performance task). 
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Illustrative Item 


Notes: Box B has a bulb in it. 


P4.11, Using Scientific Inquiry 
Source: Shavelson, Baxter, and Pine 1991, p. 357. 


Simulation items model systems (e.g., food webs), pose problems of prediction and ex- 
planation about changes in the system, and permit students to collect data and solve prob- 
lems in the system. The following screen shot comes from such a simulation ICT. 


NAEP 2015 SCIENCE FRAMEWORK 
105 


Illustrative Item 


High school (grade 12) students are asked to identify locations appropriate for solar 
power generation. (See appendix C for a description of the full task.) To complete the 
task, students must: 


Evaluate GIS map visualizations. 
Compare and contrast visualizations of different types of data. 
Use analytical extension to perform computations with visualization data. 


Following is an example of one of several tasks that a student completes: 


Your task is to identify two states that will have high annual solar energy and will be able 
to generate the maximum amount of electricity from their solar panels. Name two states 
that you predict will have a good annual electrical yield. In the rest of this performance 
assessment, you will generate visualizations and calculate which states will generate the 
best annual electricity yield from solar panels. 


The following screen shot provides an example of a student response: 


_ This visualization shows that the states with the most annual solar energy 
are Arizona and Nevada. This data alone, however, does not completely 
_ justify the location of the panels, due to temperature issues. 


E12.10, Using Technological Design 
Source: Quellmalz et al. 2004. 
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Finally, concept mapping can be done by providing concept terms and asking students to 
build propositions on the computer by linking pairs of terms with arrows and words or 
phrases. Following is a screen shot of a completed concept map. 


Illustrative Item 


In this task, middle and high school students used a customized software program to 
create concept maps. Students received 18 environmental science terms and 7 link 
labels.* Students could drag and drop these concepts onto the grid space of the mapping 
program and add, erase, and link the items in their newly constructed maps. (See 
appendix C for more information about this task.) 


[o = = Concept Mapper 
| File Add Concept Available Links 


} 


producer 


bacteria 
evaporation 


requires 
q 4 produces 


| 
causes | 
| 


influences 


carbon dioxide | 


L12.5, Identifying Science Principles 
Source: Adapted from Herl et al. 1999. 
* Link labels should not be provided to students on the NAEP Science Assessment. See the Specifications 
for more information. 


Computers and other media provide potential solutions to a variety of practical challenges 
posed by complex assessment exercises. The logistical challenges of hands-on perfor- 
mance tasks can be circumvented with computer simulation. Extensive databases can be 
used to assess students’ ability to select and evaluate information relevant to the situation 
or problem they need to address. In addition, the difficulty of providing materials and 
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training for complex tasks (such as concept maps) can be circumvented with computers. 
To avoid cheating and teaching to the concept map, concept terms can be randomly sam- 
pled for a particular map (Ruiz-Primo et al. 2001). 


ICTs should be used when the format offers advantages over other assessment modes. 
ICTs include (but are not limited to) testing student knowledge, skills, and abilities 
related to the following situations: 


e For scientific phenomena that cannot easily be observed in real time, such as see- 
ing things in slow motion (e.g., the motion of a wave) or at a higher speed (e.g., 
erosion caused by a river). They are also useful when it is necessary to freeze 
action or replay. 


e For modeling scientific phenomena that are invisible to the naked eye (e.g., the 
movement of molecules in a gas). 


e For working safely in lab-like simulations that would otherwise be hazardous 
(e.g., using dangerous chemicals) or disorderly in an assessment situation. 


e For situations that require several repetitions of an experiment in limited 
assessment time while varying the parameters (e.g., rolling a ball down a slope 
while varying the mass, the angle of inclination, or the coefficient of friction of 
the surface). 


e For searching the Internet and resource documents that provide high-fidelity 
situations related to the actual world in which such performances are likely to be 
observed. 


e For manipulating objects in a simplistic manner, such as moving concept terms in 
a concept map. 


Extended constructed-response items, concept-mapping tasks, and simulated performance 
tasks are especially strong candidates for ICTs. In this way, the complex science under- 
standings and practices that need to be probed in the NAEP Science Assessment could be 
captured with less time, cost, and logistical challenges and with greater opportunity for 
divergent problem-solving tasks than has been the case in the past. 


DISTRIBUTION OF ITEMS 


This section recommends appropriate item distributions. The three types of distribution, 
as measured by percentage of student response time at each grade level, are as follows: 


e Items by content area—Physical Science, Life Science, and Earth and Space 
Sciences 


e Items by science practice—Identifying Science Principles, Using Science Princi- 
ples, Using Scientific Inquiry, and Using Technological Design 
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e Items by type—Selected-response items and constructed-response items 


DISTRIBUTION OF ITEMS BY CONTENT AREA 


In the Overview of the Science Assessment section of this chapter, the distribution of 
items at each grade level by the three science content areas, as measured by percentage of 
student response time, was described as shown in exhibit 17. 


Exhibit 17. Distribution of items by content area and grade 


Grade 4 Grade 8 Grade 12* 
(percentage of (percentage of (percentage of 
student response | student response | student response 
time) time) time) 
Physical Science 33.3 30.0 37.5 


Earth and Space Sciences 33.3 40.0 25.0 


* These distributions are based on NAEP data regarding students’ course-taking patterns in 12th grade. If 
these patterns change substantially, the distributions should be reconsidered. 


DISTRIBUTION OF ITEMS BY SCIENCE PRACTICE 


The item distribution for the four science practices, as measured by percentage of student 
response time at each grade level, should have the following approximate allocations: 


Exhibit 18. Distribution of items by science practice and grade 


Grade 4 Grade 8 Grade 12 
(percentage of (percentage of (percentage of 
student response | student response | student response 
time) time) time) 
Identifying Science Principles 30 25 20 
Using Science Principles 30 35 40 
Using Scientific Inquiry 30 30 30 
Using Technological Design 10 10 10 


DISTRIBUTION OF ITEMS BY ITEM TYPE 


As measured by student response time, 50 percent of the assessment items at each grade 
level should be selected-response items and 50 percent should be constructed-response 
items (short constructed response, extended constructed response, concept-mapping 
tasks). Individual selected-response items and individual constructed-response items used 
within each of the item clusters, POE item sets, hands-on performance tasks, and ICTs 
should be included in this distribution. If variation from this distribution becomes neces- 
sary as items are developed, preference should be given to constructed-response items. 


In regard to the combination item types, the NAEP Science Assessment should contain at 
least one of each of the following at each grade level: item clusters, POE item sets, 
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hands-on performance tasks, and ICTs. In addition, it is highly recommended that each 
assessment include at least one concept-mapping task in grades 8 and 12. 


Hands-on performance tasks and ICTs are administered to a subset of the students sam- 
pled; the quantity for each task is specified in exhibit 19. 


Exhibit 19. Distribution of items b pe of item and grade 
Grade 4 Grade 8 Grade 12 
(number of (number of (number of 
tasks) tasks) tasks) 
Hands-On Performance Task >] > >] 


Interactive Computer Task 


Total 


In any grade, the number of ICTs should not exceed the number of hands-on performance 
tasks. No student will be administered both types of tasks. 


STUDENTS WITH DISABILITIES AND ENGLISH LANGUAGE LEARNERS 


Along with the increase in national and state testing, there is an increased demand for 
assessment systems to include all students (for example, those with disabilities and those 
learning English, many of whom have not been included in these systems in the past). As 
NAEP seeks to measure the educational progress of students in the nation’s classrooms, 
assessment developers will encounter challenges that require careful thought and consid- 
eration to the development of items that provide the most equitable context possible for 
all students. 


NAEP should strive to develop science assessments that allow the widest possible range 
of students to participate. In this way, the interpretation of scores of all participants leads 
to valid inferences about the levels of their performance as well as valid comparisons 
across states and with state assessments. All students should have the opportunity to 
demonstrate their knowledge of the concepts and ideas that the NAEP Science Assess- 
ment is intended to measure. 
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According to the National Research Council: 


Fairness, like validity, cannot be properly addressed as an afterthought 
once the test has been developed, administered, and used. It must be 
confronted throughout the interconnected phases of the testing process, 
from test design and development to administration, scoring, 
interpretation, and use (1999b, pp. 80-81). 


When assessments are first conceptualized, they must be thought of in the context of the 
entire population that will be assessed (American Educational Research Association, 
American Psychological Association, and National Council on Measurement in Educa- 
tion 1999; NRC 1999a; Thompson, Johnstone, and Thurlow 2002). NAEP assessments, 
as well as all large-scale assessments, must be responsive to growing demands: increased 
diversity, increased inclusion of all types of students in the general curriculum, and 
increased emphasis and commitment to serve and be accountable for all students. 
Assessments must measure the performance of students who have a wide range of abili- 
ties and skills, ensuring that those with diverse learning needs receive opportunities to 
demonstrate competence on the same content as all other students. 


Students with disabilities and English language learners each present challenges in how 
their science competencies can be assessed in a valid manner. Nevertheless, these groups 
share some commonalities, including considerable heterogeneity in each group regarding 
assessment needs. In addition: 


e Conceptual frameworks based on appropriate theories of language development 
and proficiency and of various forms of disabilities will be needed to build inclu- 
sive assessments. 


e A greater commitment to financial and human resources will be needed to 
develop and administer tasks and to interpret performance on tasks. 


Two general recommendations pertain to both groups in the context of good assessment 
design for all students: readability of written text and alignment to content statements. 


Students’ ability to read and respond to written text often determines successful perfor- 
mance on assessments. Assessment items may pose an unfair disadvantage for some 
students if there is a heavy burden on reading skills when reading is not the goal of the 
assessment. Language that is both straightforward and concise and that uses everyday 
words to convey meaning is needed. The goal of ensuring that language has these charac- 
teristics is to improve the comprehensibility of written text and at the same time preserve 
the essence of its meaning. Using language that reduces the linguistic demands placed on 
students reduces the effect of reading skills and language proficiency on students’ science 
performance and assessment scores. More information on reading level is provided in the 
specifications. 
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Items on the NAEP Science Assessment must be aligned to the content statements and 
science practices with the same depth and breadth of coverage and the same cognitive 
demands as specified in the framework. Assessment design should emphasize accessibil- 
ity using different formats, technologies, designs, and accommodations to include as 
many students as possible. It must be clear from the outset that, to be equitable, 
assessments need to measure the achievement of all students on the same content and 
achievement standards. 


To these ends, field tests should sample every type of student expected to participate in 
the final assessment administration, including students with a wide range of disabilities, 
English language learners, and students across racial, ethnic, and socioeconomic lines. 
Field testing NAEP items with a broad range of students will not only help determine 
whether items are unclear, misleading, or inaccessible for certain groups of students, but 
will also help ensure that assessment procedures are accessible to students when the 
NAEP Science Assessment is fully implemented. More information about these recom- 
mendations can be found in the Specifications. 


NAEP strives to assess all students selected by its sampling process. Rigorous criteria are 
applied to minimize the number of English language learners and students with disabili- 
ties who are excluded from NAEP assessments. Participating students with special needs 
are permitted to use accommodations, as stated in current NAEP policy: 


All special-needs students may use the same accommodations in NAEP 
assessments that they use in their usual classroom testing unless the ac- 
commodation would make it impossible to measure the ability, skill, or 
proficiency being assessed, or the accommodation is not possible for the 
NAEP program to administer (NCES 2005a). 


See the Specifications for more information on NAEP’s inclusion policy and permitted 
accommodations. 


SPECIAL STUDIES 


Special studies bearing on aspects of the NAEP Science Assessment are presented in this 
framework. Each study contributes to a further understanding of science assessment. 


Group 1 special studies have the highest priority: 


e “Exchangeability” of hands-on performance and interactive computer 
investigations 


e Impact of variation in item format and language demand on the performance of 
English language learners and students with disabilities 


e Computer adaptive testing to assess the development of student understanding of 
Earth systems 
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Group 2 special studies have lower priority: 
e Knowing what students know about technological design 
e Extended investigations by students 


The order in which studies are listed does not imply priority within group 1 or group 2. 
Group 1 special studies are described in the next section. Group 2 special studies are dis- 
cussed in appendix D. 


GRouUP 1 SPECIAL STUDIES 


“Exchan eability” of Hands-on Performance and Interactive Computer Investiga- 
tions 


Inquiry is at the heart of knowing and doing science. A fundamental aspect of inquiry is 
the design, conduct, and interpretation of empirical investigations to answer a question or 
test a hypothesis. Although a full assessment of inquiry is not possible on any test that is 
given on demand, hands-on performance investigations (HPIs) attempt to approximate 
this aspect of inquiry under time, space, cost, and logistical constraints. For this reason, 
HPIs have been a part of the NAEP Science Assessment since 1996. 


However, HPIs have been criticized as being expensive, logistically difficult, and too 
highly structured. On the other hand, ICTs or, in this case, interactive computer investiga- 
tions (ICIs), are logistically simpler, less expensive, and less structured. Therefore, the 
purpose of this study is to explore whether ICIs and HPIs are exchangeable. The question 
is not whether ICIs could replace HPIs either on NAEP or in the classroom—they should 
not. Even if these two approaches produce similar performances and scores, each affords 
somewhat different opportunities; simulations cannot be exchanged with actual practice. 
This is a question of assessment: can the cost and logistical challenges of HPIs be 
reduced with the use of ICIs and still measure the same competencies with as much 
reliability and validity? Some research suggests that the two methods of assessing 
student inquiry are exchangeable to a fair degree (e.g., Pine, Baxter, and Shavelson 1993; 
Rosenquist, Shavelson, and Ruiz-Primo 2000). However, further research is needed on 
several different investigations to provide a satisfactory answer for large-scale assess- 
ment. 


Specifically, this study would address the following research questions: 
e Does the choice of an ICI or HPI limit the questions that may be asked? 
Specifically, is there something of value in an HPI that cannot be asked if an ICI 
is administered? 


e Arescores on HPIs and ICIs equally reliable? 


e Are scores on HPIs and ICIs of equal magnitude? 
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e To what extent does performance on an HPI predict performance on an ICI in the 
same investigation? 


e Doscores on HPIs and ICIs correlate somewhat equally with scores on another 
measure of science inquiry or achievement? 


e Are similar thinking processes evoked by HPIs and ICIs? 


e Do the answers to these questions depend on individual differences among 
students (such as gender, English proficiency, race/ethnicity, socioeconomic sta- 
tus, and geographic location)? Variation in student access to computers is also an 
area of interest; this factor could be confused with other variables listed here. 


Impact of Variation in Item Format and Language Demand on the Performance of 
English Language Learners and Students With Disabilities 


English language learners and students with disabilities do not tend to perform as well on 
standardized achievement assessments as other students. Recent studies, for example, 
have pointed to a systematic relationship between the linguistic complexity of the 
assessment and the test scores of English language learners (e.g., Abedi 2003) and 
students with certain disabilities, such as those related to reading and information pro- 
cessing. Science assessments, with their heavy reliance on verbal skills, may exacerbate 
performance disparities. In cases where this relationship is demonstrable and test items 
use complex language, the differences become sources of measurement error and con- 
struct irrelevant variance; therefore, the nature of the assessment item must be addressed. 
Until this dimension of the assessment item is more clearly understood, any interpretation 
of the performance of English language learners or students with disabilities on a content 
assessment is problematic; for example, language proficiency and science understanding 
cannot be separated. 


Preliminary results from several studies of scaffolded science assessments that are 
designed to minimize language complexity and provide alternative response modalities— 
including graphic organizers or drawn representations of the concepts—indicate that 
research is needed to clarify the relationship between language complexity, scaffolded 
assessment items, and the performance of English language learners and students with 
disabilities. 


Specifically, this study would address the following research questions: 


e Can the language complexity of a content-based assessment be systematically 
measured? 


e Can content-based assessment items be designed to minimize the language 
demand while conserving the content information obtained? 


e Ifthe content-based assessment contains a graphic response modality, do English 
language learners and students with disabilities demonstrate higher understanding 
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of the content concept being assessed relative to more linguistically demanding 
response modalities? 


e When the content-based assessment with a graphic response option is also com- 
puter based, is there a further benefit in terms of content concept conservation and 
the performance of these students? 


Computer Adaptive Testing To Assess the Development of Student Understanding 
of Earth Systems"! 


A common critique of large-scale assessment is that its necessary reliance on easily 
scored, decontextualized, and simplified items has led to an impoverished range of poten- 
tial learning activities from which valid and reliable measures might be derived (Resnick 
and Resnick 1992). Attempts to find alternatives include the following: 


e The facets approach (Minstrell 1998), which posits a strong model of facets of 
student knowledge for certain science topic areas and uses coordinated sets of 
multiple-choice items to hone in on students’ particular conceptions and miscon- 
ceptions; and 


e The progress variable approach (Masters, Adams, and Wilson 1990), which posits 
a learning progression and uses item response theory to scale students’ responses 
to (typically open-ended) items to estimate the part of the learning progression in 
which students will most likely be located. 


This special study combines the strengths of each of these approaches to develop a new 
type of “branching” item that can be used to investigate the more complex types of 
knowledge structures and complex procedural steps involving contingencies such as 
those common in inquiry-related contexts, and yet maintain the efficiency of traditional 
multiple-choice testing. Specifically, the facets approach will contribute its strong 
knowledge structure and convenient scoring, and the progress variables approach will 
contribute the interpretational framework of the learning progression and the flexible sta- 
tistical modeling available through recent advances in item response modeling (De Boeck 
and Wilson 2004). Together, these approaches allow the use of item bundles (such as 
those shown in exhibit 20) to provide both the usual result in terms of student ability es- 
timation as well as potentially more educationally informative results such as the preva- 
lence of particular classes of misconceptions among the student body. 


" Currently, a number of states are using computerized testing, consisting largely of translating traditional 
paper-and-pencil items into computer-based delivery systems. The study suggested here, as well as the first 
study on p. 114, makes more extensive use of the capabilities inherent in computer-based assessments. 
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Exhibit 20. Storyboard showing item design for a 
“branching” item bundle on ions and atoms (adapted from Scalise 2004) 


Matter Composition: lons and Atoms Item Bundle 


1, Lead-based paint contains Pb2+ ions and lead pipes are made up of Pb atoms 
The main difference between Pb2+ ion and Pb atom is: 

A.They are basically the same. (go to question 2) 

B. They have a different number of electrons. (go to question 3) 

C. They have a different number of protons (3) 

D.They are different but not in the ways described. (go to question 2) 


E.| don't know (0) 
2. Choose the answer with which you most agree. 3. Pick the best answer below: 
Pb2+ ion and Pb atom are the same except: A.Pb2+ has 2 fewer electrons than Pb. 
A.Pb2+ has ionic bond and Pb has atomic bonds. (3) (go to question 4) 
B.Pb2+ ion and Pb are similar but used differently. (1) B.Pb2+ has a larger e- density cloud. (3) 
C.Pb2+ is a liquid, Pb is a sold. (2) C-Pb2+ is positively charged so has 
D.Pb2+ requires two Pb atoms. (2) 2 extra valence electrons. (4) 


4, Electrons can be thought of as arranged in shells or orbitals. 
Select the ground state electron configuration for carbon: 
A.1s22s12p3 (7) 

B. 1522s22p2 (go to question 5) 

C.1s22s22p1 (6) 

D.None of these (5) 


5. Certain main group elements exhibit multiple oxidation states, including Group 14 
of which carbon is a member. It would seem that a possible oxidation state for carbon 
is +2, with two electrons available for bonding in half-filled 2p, and 2py orbitals. Based 
on valence bond theory, show an electron configuration justifying the formation 

of four bonds for carbon rather than two and describe the type of orbitals formed. 
EXPLAIN your answer as fully as possible. 

(All respondents who reach this screen will be scored as an 8. 

Answers submitted here will be used to build distractors for this question.) 


Possible scores and parameters for this item bundle under iota model (assume constraint on cases): 
score 0 1 2 3 4 5 6 7 8 
# paths 1 1 2 3 1 1 1 1 


1 
parameters 67, 572 = 873.1731 = 74,1741.1742 575 876 577 57g 579 
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Specifically, this study would address the following research questions: 


e Can the “branching” item type be developed and delivered in a logistically 
efficient way for use in NAEP? 


e Can the information from sets of “branching” item bundles be used to provide 
reliable, valid, and useful information on both overall student ability in science 
and the classification of students into educationally useful categories? 


The study would focus on a specific Earth system that is of practical and environmental 
significance, such as the biogeochemical carbon cycle. Understanding this system and 
related environmental issues (e.g., global climate change) requires connected understand- 
ings in the Physical, Life, and Earth and Space Sciences, many of which are characterized 
in the framework. For example, students who understand global warming understand how 
photosynthesis, cellular respiration, and fossil fuel combustion affect carbon dioxide con- 
centrations in the atmosphere. This connected understanding can be tracked as a learning 
progression. 


While these three specific studies were not conducted due to resource constraints, NAEP 
continues to pursue a broad program of research in areas related to innovative assessment 
tasks, testing special needs students, and computer adaptive testing. 
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APPENDIX A 
STEERING COMMITTEE GUIDELINES 


THE FRAMEWORK IS INFORMED BY THE NATIONAL STANDARDS AND 
BENCHMARKS 


The framework should reflect the nation’s best thinking in science instruction; therefore, 
it is guided by two national documents: National Science Education Standards (NRC 
1996) and Benchmarks for Science Literacy (AAAS 1993). Both documents were subject 
to extensive internal and external reviews during their development. 


Informed by the National Standards and Benchmarks, the framework should emphasize 
knowledge and use of science concepts, appropriate linking of science facts to concepts, 
relationships among concepts, and major themes unifying the sciences. The framework 
should also incorporate investigative skills. 


THE FRAMEWORK REFLECTS THE NATURE AND PRACTICE OF SCIENCE 


The National Standards and Benchmarks include standards addressing science as inquiry, 
nature of science, history of science, and the designed world. The framework should 
emphasize the importance of these aspects of science education and should include the 
expectation that students will understand the nature and practice of science. Science is a 
self-correcting process; that is, a way of knowing in which theories are continually modi- 
fied and refined based on new research findings. Students should demonstrate the ability 
to accomplish the following: 


e Make warranted inferences from evidence. 
e Use evidence to justify conclusions based on scientific investigations. 


e Demonstrate reasoning skills in the application of science content and in under- 
standing the connections between science concepts. 


e Exercise skepticism when evaluating, using, and discarding data. 
e Understand and use models to describe and do science. 


e Apply content knowledge and skills to solve problems as they occur in the natural 
world. 
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e Understand and apply knowledge of links and commonalities of science across 
fields. 


The scientific disciplines are no longer practiced in isolation, and research that cuts 
across discipline boundaries is common. The framework should: 


e identify some of the science concepts and skills that cut across the assessed con- 
tent areas; 


e address science in both the natural and designed world; and 


e clearly define and identify commonalities and differences between “science” and 
“technology” or “technological design.” 


The framework should also address social and historical contexts, which are keys to 
understanding how the scientific community has arrived at its current body of knowledge. 


THE FRAMEWORK INCORPORATES KEY ATTRIBUTES OF EFFECTIVE 
ASSESSMENT 


The framework should use assessment formats that are consistent with the objectives 
being assessed. It should be guided by the best available research on assessment item de- 
sign and delivery. 


The framework should take into account student diversity as reflected in gender, 
geographic location, language proficiency, race/ethnicity, socioeconomic status, and dis- 
ability condition. The assessment should be designed and written so it is accessible by the 
majority of students and to minimize the need for special accommodations for both 
students with disabilities and English language learners. Students with special needs 
should be provided accommodations so they can participate in the assessment. 


The framework should reflect knowledge about the acquisition of key science concepts 
over time, based on research about how students learn. The existing research findings 
should make clear, when possible, the progression of science knowledge across the grade 
levels. Concepts should be represented in a manner that reflects how students progress 
through a discipline and across disciplines. Assessment items should reflect students’ po- 
tential for applying concepts and more varied and complex situations over time. 


Critical content and skills should be articulated and assessed across grades 4, 8, and 12 
(vertically) and across the fields of science (horizontally) by creating items that are 
deliberately layered to achieve these goals. An example of measuring similar constructs 
within and across subjects is the progression of increasingly sophisticated understanding 
about energy from elementary school to middle school to high school in the content areas 
of biology, chemistry, Earth science, and physics. 
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The NAEP assessment should continue to use a variety of assessment formats, including 
well-constructed multiple-choice and open-ended items as well as performance tasks. In 
addition, multiple methods of assessment delivery should be considered, including the 
appropriate uses of digital-based technology. The framework should consider using digi- 
tal delivery systems for the assessment, including Web-based or CD-ROM formats. Em- 
bedded simulations that can represent scientific phenomena such as data, representations, 
and factors captured within laboratory experiments, along with the use of an adaptively 
designed series of assessment items, should also be considered. Advances in machine 
scoring of text should provide the opportunity for increased use of open-response format 
questions. The assessment format and delivery system employed should be accessible to 
the widest range of students. 


Each achievement level—Basic, Proficient, and Advanced—should include a range of 
items assessing various levels of cognitive knowledge that is broad enough to ensure each 
is measured with the same degree of accuracy. Descriptions of Basic, Proficient, and Ad- 
vanced must be as clear as possible. 


THE ASSESSMENT PROVIDES DATA FOR RESEARCH 


NAEP assessment results are increasingly being used to review state student assessments 
and compare student achievement across states. The framework should address the 
important uses of assessment data both to conduct research to better understand science 
learning and to improve science achievement. Data from the assessment should be col- 
lected in such a way as to provide information that accomplishes the following: 


e supplies details of the attributes (e.g., race/ethnicity, gender) of the students being 
assessed; 


e provides results by student gender, race/ethnicity, and socioeconomic level; 


e describes the academic preparation of the teachers whose students are being as- 
sessed; 


e describes the nature of the educational system of the students being assessed; 
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e relates the instructional delivery and materials, professional development of the 
teachers, and the learning environment to the results from the assessment; and 


e provides feedback to educators for improving science instruction and learning. 


THE SPECIFICATIONS DOCUMENT IS CLOSELY ALIGNED WITH THE 
FRAMEWORK 


The connections among the framework, the specifications, and the assessment items 
should be transparent, have a consistent level of specificity, and be coherent. 


The specifications should be written with consistent detail across all fields, domains, and 
expectations of the framework: 


e The specifications should have a consistent structure across all areas. 


e The expected science knowledge that represents the target for assessment should 
be described in a clear and consistent format. The specifications should address 
content that reflects the standards and focuses on the significant information and 
knowledge that students should retain over time (e.g., principal ideas, fundamen- 
tal understandings). 


e The specifications should use verbs that describe the expected action to be taken 
in the assessment (e.g., identify, describe, evaluate, relate, analyze, demonstrate). 


e Expectations across the content areas should match in level of specificity and 
scope. 


e The specifications should follow the idea of learning trajectories. To assess over- 
arching concepts or themes, the assessment specifications should reflect a layered 
understanding of growth in knowledge of the concepts. 
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APPENDIX B 
NAEP SCIENCE ACHIEVEMENT LEVEL DESCRIPTIONS 


Congress authorized the Governing Board to develop appropriate student achievement 
levels on NAEP. The achievement level descriptions are statements of what students 
should know and be able to do on NAEP at grades 4, 8, and 12. To fulfill its statutory 
responsibility, the Governing Board developed a policy to guide the development of 
achievement levels for all NAEP subjects. It identified three levels of achievement to 
provide the public, educators, and policymakers with information on student performance 
on NAEP. These levels—Basic, Proficient, and Advanced—are used as a primary means 
of reporting NAEP results to describe “how good is good enough” at grades 4, 8, and 12. 


Exhibit B-1 displays the Governing Board’s generic policy definitions for Basic, 
Proficient, and Advanced achievement that pertain to all NAEP subjects and grades. 


Exhibit B-1. Generic achievement level policy definitions for NAEP 


Achievement Policy Definition 
Level 
Advanced This level signifies superior performance. 


This level represents solid academic performance for each grade 
assessed. Students reaching this level have demonstrated 
Proficient competency over challenging subject matter, including subject- 
matter knowledge, application of such knowledge to real-world 
situations, and analytical skills appropriate to the subject matter. 


This level denotes partial mastery of prerequisite knowledge and 


poe skills that are fundamental for proficient work at each grade. 


During the framework development process, the project committees are asked to develop 
preliminary achievement level descriptions, based on the generic policy definitions, to 
guide item development. The descriptions are designed to provide examples of what 
students performing at the Basic, Proficient, and Advanced achievement levels should 
know and be able to do in terms of the science content and practices identified in the 
framework; they are used to ensure that a broad range of items (covering Basic, Profi- 
cient, and Advanced levels) is developed at each grade level. The NAEP assessment 
development contractor and item writers are the intended audiences for the descriptions. 
Planning committee members used their expert judgment to draft the preliminary 
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descriptions and were partially guided by the following considerations: 


e Basic descriptions could be composed of both moderately challenging content and 
practice components. 

e Proficient descriptions could be composed of a mixture of highly and moderately 
challenging content and practice components. 

e Advanced descriptions could be composed of highly challenging science content 
and highly challenging components of a given science practice. 


The preliminary descriptions include only illustrative statements drawn from the frame- 
work’s science content and practices. The statements are not intended to represent the 
entire set of objectives from the content and practice dimensions, nor do the preliminary 
achievement level descriptions denote a sense of priority or importance based on the 
statements selected. In addition, the descriptions should not be interpreted as assessment 
items or as suggestions for assessment items. Although adjectives are not explicitly 
included in every statement, each was written with the expectation that student perfor- 
mances would be “good” and “reasonable.” For example, the phrase “design an investiga- 
tion” implies “design an organized and logical investigation” and “propose a framework” 
implies “propose a rational framework.” 


The achievement level descriptions are a set of paragraphs derived from the preliminary 
achievement level descriptions, and these paragraphs are used to report the NAEP science 
results to the general public and other audiences. For each grade level, the paragraphs 
describe what students should know and be able to do at the Basic, Proficient, and 
Advanced levels in terms of the science content and practices identified in the 
framework. After the assessment is administered, broadly representative panels engage in 
a standard-setting process to determine the achievement level cut scores on the NAEP 
scale. The cut scores represent the minimum score required for performance at each 
NAEP achievement level. 
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NAEP SCIENCE ACHIEVEMENT LEVEL DESCRIPTIONS 


The achievement levels are cumulative; therefore, students performing at the Proficient level 
also display the competencies associated with the Basic level, and students at the Advanced 
level also demonstrate the skills and knowledge associated with both the Basic and the Profi- 
cient levels. The cut score indicating the lower end of the score range for each level is noted 
in parentheses. Bold text is a short, general summary to describe performance at each 
achievement level. 


NAEP SCIENCE ACHIEVEMENT LEVELS—GRADE 4 


Basic 


Students performing at the Basic level should be able to describe, 
measure, and classify familiar objects in the world around them, as 
well as explain and make predictions about familiar processes. 
These processes include changes of states of matter, movements of 
objects, basic needs and life cycles of plants and animals, changes in 
shadows during the day, and changes in weather. They should be 
able to critique simple observational studies, communicating obser- 
vations and basic measurements of familiar systems and processes, 
and look for patterns in their observations. With regard to scientific 
constraints, they should also be able to propose and critique alter- 
native solutions to problems involving familiar systems and pro- 
cesses. 


Science Practices: Students performing at the Basic level should be able 
to describe, measure, and classify familiar objects in the world around 
them, as well as explain and make predictions about familiar processes, 
using evidence to support their observations and conclusions. They 
should be able to critique simple observational studies, communicate 
observations and basic measurements of familiar systems and processes, 
and look for patterns in their observations. They should also be able to 
propose and recognize alternative solutions to problems involving fa- 
miliar systems and processes. 


In the physical sciences, students performing at the Basic level should 
be able to describe the properties of the states of matter, describe how to 
change matter from one state to another, describe different forms of en- 
ergy, predict the electrical energy transfers that will take place in a sim- 
ple circuit, critique alternative explanations for changes in a moving ob- 
ject’s position, and design an investigation to show how exerting a force 
on an object changes the object’s motion. 


In the life sciences, students performing at the Basic level should be able 
to identify the stages in the life cycles of familiar organisms; describe 
how familiar animals meet their basic needs for food, air, water, and 
shelter; observe and describe the changes in plants and animals during 
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Proficient 


their life cycles; and describe how environments meet the survival needs 
of familiar plants and animals. 


In the Earth and space sciences, students performing at the Basic level 
should be able to predict changes in the length and position of shadows 
cast by the sun, describe how slow Earth processes (e.g., erosion) and 
fast Earth processes (e.g., volcanic eruption) can change Earth’s surface, 
distinguish between natural and manmade materials, choose and use a 
tool to monitor how weather conditions change, and identify Earth re- 
sources that are limited. 


Students performing at the Proficient level should be able to demon- 
strate relationships among closely related science concepts, as well 
as analyze alternative explanations or predictions. They should be 
able to explain how changes in temperature cause changes of state, 
how forces can change motion, how adaptations help plants and an- 
imals meet their basic needs, how environmental changes can affect 
their growth and survival, how land formations can result from 
Earth processes, and how recycling can help conserve limited re- 
sources. They should be able to identify patterns in data and/or ex- 
plain these patterns. They should also be able to identify and cri- 
tique alternative responses to design problems. 


Science Practices: Students performing at the Proficient level should be 
able to demonstrate relationships among closely related science con- 
cepts and familiar phenomena around them, as well as analyze alterna- 
tive explanations or predictions, using evidence to support their expla- 
nations and predictions; critique observational studies and simple inves- 
tigations; identify patterns in data and/or explain those patterns in data; 
and apply scientific ideas to identify and critique alternative designs to 
problems that personally affect them. 


In the physical sciences, students performing at the Proficient level 
should be able to demonstrate the relationship between temperature 
change and changes in the physical properties of matter, explain how 
energy in one form can be changed into another form, design an investi- 
gation that measures how temperature changes when energy is added to 
a substance, propose a design for a container that will maintain the tem- 
perature of an object that is above or below room temperature, and 
measure changes in position of an object in motion as different forces 
are applied. 


In the life sciences, students performing at the Proficient level should be 
able to describe needs of familiar plants and animals at different stages 

of their life cycles, explain adaptations of familiar plants and animals to 
their environments, predict effects of environmental changes on plant or 
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Advanced 


animal growth and survival, and apply information about an animal’s 
basic needs to propose a supportive environment. 


In the Earth and space sciences, students performing at the Proficient 
level should be able to explain how the Sun’s changing position in the 
sky during the day affects shadows; interpret land formations as result- 
ing from either slow (e.g., erosion) or rapid (e.g., volcanic eruption) 
Earth processes; explain how natural materials can help sustain the lives 
of familiar plants and animals; identify how patterns of weather condi- 
tions change from season to season; and explain how the practices of 
recycling, reusing, and reducing help to conserve limited resources. 


Students performing at the Advanced level should be able to demon- 
strate relationships among different representations of science prin- 
ciples, as well as propose alternative explanations or predictions of 
phenomena. They should be able to use numbers, drawings, and 
graphs to describe and explain motions of objects; analyze how en- 
vironmental conditions affect growth and survival of plants and an- 
imals; describe changes in the Sun’s path through the sky at differ- 
ent times of year; and describe how human uses of Earth materials 
affect the environment. They should be able to design studies that 
use sampling strategies to obtain evidence. They should also be able 
to propose and critique alternative individual and local community 
responses to design problems. 


Science Practices: Students performing at the Advanced level should be 
able to demonstrate relationships among different representations of 
principles, as well as propose alternative explanations or predictions of 
familiar phenomena, using evidence to support their explanations and 
predictions; design observational studies or simple investigations to val- 
idate or criticize explanations or predictions and use sampling strategies 
to obtain evidence; and propose and critique alternative individual and 
local community responses to design problems. 


In the physical sciences, students at the Advanced level should be able 
to demonstrate the relationship between the quantity of energy needed 
to change the state of a sample of a substance and the weight of the 
sample, demonstrate how different representations (i.e., verbal, numeri- 
cal, graphical) can be used to show the motion of an object, suggest an 
example of how the motion of an object can be changed without touch- 
ing it, and design an investigation that demonstrates how long it takes 
different forms of energy to change the temperature of matter. 


In the life sciences, students at the Advanced level should be able to 
evaluate relationships between changing environmental conditions and 
organisms’ growth, survival, and reproduction; analyze environments 
for how they may have different effects on the growth and survival of 
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plants or animals of the same kind; and investigate the relationship be- 
tween light and plant growth. 


In the Earth and space sciences, students at the Advanced level should 
be able to relate changes in the Sun’s daily path through the sky to dif- 
ferent times of year, suggest examples of Earth materials that can be 
modified to meet human needs, explain how erosion is caused by dai- 
ly/seasonal weather events, propose methods of reducing the amount of 
erosion, describe how humans can change environments that can be ei- 
ther detrimental or beneficial for themselves and other organisms, and 
describe how the use of Earth materials by humans impacts the envi- 
ronment. 


NAEP SCIENCE ACHIEVEMENT LEVELS—GRADE 8 


Basic 


Students performing at the Basic level should be able to state or 
recognize correct science principles. They should be able to explain 
and predict observations of natural phenomena at multiple scales, 
from microscopic to global. They should be able to describe proper- 
ties and common physical and chemical changes in materials; de- 
scribe changes in potential and kinetic energy of moving objects; 
describe levels of organization of living systems—cells, multicellular 
organisms, and ecosystems; identify related organisms based on he- 
reditary traits; describe a model of the solar system; and describe 
the processes of the water cycle. They should be able to design ob- 
servational and experimental investigations employing appropriate 
tools for measuring variables. They should be able to propose and 
critique the scientific validity of alternative individual and local 
community responses to design problems. 


Science Practices: Students performing at the Basic level should be able 
to state or recognize correct science principles; explain and predict ob- 
servations of natural phenomena at multiple scales, from microscopic to 
global, using evidence to support their explanations and predictions; de- 
sign investigations employing appropriate tools for measuring variables; 
and propose and critique the scientific validity of alternative individual 
and local community responses to design problems. 


In the physical sciences, students at the Basic level should be able to 
recognize a class of chemical compounds by its properties; design an 
investigation to show changes in properties of reactants and products in 
a chemical process such as burning or rusting; describe the changes in 
kinetic and potential energy of an object such as a swinging pendulum; 
describe and compare the motions of two objects moving at different 
speeds from a table of their position and time data; describe the direc- 
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Proficient 


tion of all forces acting on an object; and suggest an example of a sys- 
tem in which forces are acting on an object but the motion of the object 
does not change. 


In the life sciences, students at the Basic level should be able to identify 
levels of organization within cells, multicellular organisms, and ecosys- 
tems; describe how changes in an environment relate to an organism’s 
survival; describe types of interdependence in ecosystems; identify re- 
lated organisms based on hereditary traits; discuss the needs of animals 
and plants to support growth and metabolism; and analyze and display 
data showing simple patterns in population growth. 


In the Earth and space sciences, students at the Basic level should be 
able to describe a Sun-centered model of the solar system that illustrates 
how gravity keeps the objects in regular motion; describe how fossils 
and rock formations can be used as evidence to infer events in Earth’s 
history; relate major geologic events, such as earthquakes, volcanoes, 
and mountain building to the movement of lithospheric plates; use 
weather data to identify major weather events; and describe the process- 
es of the water cycle including changes in the physical state of water. 


Students performing at the Proficient level should be able to demon- 
strate relationships among closely related science principles. They 
should be able to identify evidence of chemical changes; explain and 
predict motions of objects using position-time graphs; explain me- 
tabolism, growth, and reproduction in cells, organisms, and ecosys- 
tems; use observations of the Sun, Earth, and Moon to explain visi- 
ble motions in the sky; and predict surface and groundwater 
movements in different regions of the world. They should be able to 
explain and predict observations of phenomena at multiple scales, 
from microscopic to macroscopic and local to global, and to suggest 
examples of observations that illustrate a science principle. They 
should be able to use evidence from investigations in arguments that 
accept, revise, or reject scientific models. They should be able to use 
scientific criteria to propose and critique alternative individual and 
local community responses to design problems. 


Science Practices: Students performing at the Proficient level should be 
able to demonstrate relationships among closely related science princi- 
ples; explain and predict observations of phenomena at multiple scales, 
from microscopic to macroscopic and local to global, and to suggest ex- 
amples of observations that illustrate a science principle; design investi- 
gations requiring control of variables to test a simple model, employing 
appropriate sampling techniques and data quality review processes, and 
use the evidence to communicate an argument that accepts, revises, or 
rejects the model; and propose and critique solutions and predict the sci- 
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Advanced 


entific validity of alternative individual and local community responses 
to design problems. 


In the physical sciences, students at the Proficient level should be able to 
demonstrate the relationship between the properties of chemical ele- 
ments and their position on the periodic table; use empirical evidence to 
demonstrate that a chemical change has occurred; demonstrate the rela- 
tionship of the motion of an object that experiences multiple forces with 
the representation of the motion on a position-time graph; predict the 
position of a moving object based on the position-time data presented in 
a table; and suggest examples of systems in which potential energy is 
converted into other forms of energy. 


In the life sciences, students at the Proficient level should be able to ex- 
plain metabolism, growth, and reproduction at multiple levels of living 
systems: cells, multicellular organisms, and ecosystems; predict the ef- 
fects of heredity and environment on an organism’s characteristics and 

survival; use sampling strategies to estimate population sizes in ecosys- 
tems; and suggest examples of sustainable systems for multiple organ- 

isms. 


In the Earth and space sciences, students at the Proficient level should 
be able to explain how gravity accounts for the visible patterns of mo- 
tion of the Earth, Sun, and Moon; explain how fossils and rock for- 
mations are used for relative dating; use models of Earth’s interior to 
explain lithospheric plate movement; explain the formation of Earth ma- 
terials using the properties of rocks and soils; identify recurring patterns 
of weather phenomena; and predict surface and groundwater movement 
in different regions of the world. 


Students performing at the Advanced level should be able to develop 
alternative representations of science principles and explanations of 
observations. They should be able to use information from the peri- 
odic table to compare families of elements; explain changes of state 
in terms of energy flow; trace matter and energy through living sys- 
tems at multiple scales; predict changes in populations through nat- 
ural selection and reproduction; use lithospheric plate movement to 
explain geological phenomena; and identify relationships among 
regional weather and atmospheric and ocean circulation patterns. 
They should be able to design and critique investigations involving 
sampling processes, data quality review processes, and control of 
variables. They should be able to propose and critique alternative 
solutions that reflect science-based trade-offs for addressing local 
and regional problems. 


Science Practices: Students performing at the Advanced level should be 
able to demonstrate relationships among different representations of sci- 
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ence principles. They should be able to explain and predict observations 
of phenomena at multiple scales, from microscopic to macroscopic and 
local to global, and develop alternative explanations of observations, 
using evidence to support their thinking. They should be able to design 
control of variable investigations employing appropriate sampling tech- 
niques and data quality review processes that strengthen the evidence 
used to argue for one alternate model over another. They should be able 
to propose and critique alternative solutions that reflect science-based 
trade-offs for addressing local and regional problems. 


In the physical sciences, students at the Advanced level should be able 
to interpret diagrams, graphs, and data to demonstrate the relationship 
between the particulate nature of matter and state changes (for instance, 
melting and freezing); demonstrate relationships between position on 
the periodic table and the characteristics of families of the chemical el- 
ements; explain changes of state in terms of energy flow in and out of a 
system; identify possible scientific trade-offs in making decisions on the 
design of an electrical energy power plant; suggest examples of systems 
in which objects are undergoing transitional, vibrational, and rotational 
motion; and suggest examples of systems in which forces are acting 
both through contact and at a distance. 


In the life sciences, students at the Advanced level should be able to ex- 
plain movement and transformations of matter and energy in living sys- 
tems at cellular, organismal, and ecosystem levels; predict changes in 
populations through natural selection and reproduction; and describe an 
ecosystem’s populations and propose an analysis for changes based on 
energy flow through the system. 


In the Earth and space sciences, students at the Advanced level should 
be able to explain the seasons, Moon phases, and lunar and solar eclip- 
ses; illustrate how fossils and rock formations can provide evidence of 
changes in environmental conditions over time; use lithospheric plate 
movement to explain geological phenomena; identify relationships 
among regional weather and atmospheric and ocean circulation patterns; 
and use the water cycle to propose and critique ways for obtaining 
drinkable water. 


NAEP SCIENCE ACHIEVEMENT LEVELS—GRADE 12 


Basic 


Students performing at the Basic level should be able to describe, 
measure, classify, explain, and predict phenomena at multiple 
scales, from atomic/molecular to interstellar. These phenomena in- 
clude the structure of atoms and molecules; transformations of mat- 
ter and energy in physical, Earth, and living systems; motions of 
objects; the genetic role of DNA; changes in populations and ecosys- 
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Proficient 


tems due to selection pressures; earthquakes and volcanoes; pat- 
terns in weather and climate; and biogeochemical cycles. They 
should be able to design and critique observational and experi- 
mental studies, and they should be able to propose and critique so- 
lutions to problems at local or regional scales. 


Science Practices: Students performing at the Basic level should be able 
to describe, measure, classify, explain, and predict phenomena at multi- 
ple scales, from atomic/molecular to cosmic; design and critique obser- 
vational and experimental studies, controlling single variables, making 
basic decisions about sampling, analyzing reliability of data, and using 
scientific models to explain results; and propose, critique, and predict 
scientific outcomes of responses to problems at local or regional scales. 


In the physical sciences, students at the Basic level should be able to 
explain the differences in atomic structure across families in the period- 
ic table and explain how the structures of molecules change in chemical 
reactions; distinguish linear velocity and acceleration as each is repre- 
sented graphically and suggest ways in which forces can be measured; 
critique data that claim to show how gravitational potential energy 
changes with distance from the Earth’s surface; predict the situations in 
which a net force changes the motion of an object; and predict how the 
energy packets of electromagnetic waves change as the frequency of the 
waves change. 


In the life sciences, students at the Basic level should be able to identify 
changes in populations due to selection pressures and trace matter and 
energy through organisms and ecosystems; explain changes in ecosys- 
tem structure and function and identify ways in which humans can per- 
manently alter ecosystems through intentional design or unintended 
consequences; and describe the relationship between DNA and an indi- 
vidual’s hereditary traits. 


In the Earth and space sciences, students at the Basic level should be 
able to describe a Sun-centered model of the solar system that illustrates 
how gravity keeps objects in regular motion; describe how fossils and 
rock formations can be used as evidence to infer events in Earth’s histo- 
ry; relate major geologic events, such as earthquakes, volcanoes, and 
mountain building to the movement of lithospheric plates; use weather 
data to identify major weather events; and describe the processes of the 
water cycle, including changes in the physical state of water. 


Students performing at the Proficient level should be able to demon- 
strate relationships and compare alternative models, predictions, 
and explanations. They should be able to explain trends among ele- 
ments in the periodic table; conservation laws; chemical mecha- 
nisms for metabolism, growth, and reproduction; changes in popu- 
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lations due to natural selection; the evolution of the Universe; and 
evidence for boundaries and movements of tectonic plates. They 
should be able to design and critique observational and experi- 
mental studies, controlling multiple variables, using scientific mod- 
els to explain results, and choosing among alternative conclusions 
based on arguments from evidence. They should be able to compare 
scientific costs or risks and benefits of alternative solutions to prob- 
lems at local or regional scales. 


Science Practices: Students performing at the Proficient level should be 
able to describe, measure, classify, explain, and predict phenomena at 
multiple scales, from atomic/molecular to cosmic; demonstrate relation- 
ships and compare alternative models, predictions, and explanations; 
design and critique observational and experimental studies, controlling 
multiple variables, making basic decisions about sampling, analyzing 
reliability of data, using scientific models to explain results, and choos- 
ing among alternative conclusions based on arguments from evidence; 
and compare scientific costs or risks and benefits of alternative solutions 
to problems at local or regional scales. 


In the physical sciences, students at the Proficient level should be able 
identify the unique properties of water and their implications for Earth’s 
organisms and climate; describe the pattern of data expected within a 
family of elements from the periodic table; predict the nature of an un- 
balanced force on an object by the object’s motion and describe obser- 
vations that would imply a conservation principle in science; suggest 
examples of how energy gets transferred in different processes; and de- 
sign an experiment that will yield the average speed of an object under a 
free-fall situation. 


In the life sciences, students at the Proficient level should be able to ex- 
plain chemical mechanisms for metabolism, growth, and reproduction in 
living systems; analyze cases of evolutionary change in populations us- 
ing the following related science principles: the potential of a species to 
increase its numbers, the genetic variability of its offspring, limitations 
on the resources required for life, and the ensuing selection of those or- 
ganisms better able to survive and leave offspring; and use scientific 
models to explain data patterns related to metabolism, genetics, or 
changes in ecosystems. 


In the Earth and space sciences, students at the Proficient level should 
be able to describe the theory that the Universe expanded from a single 
point billions of years ago and that most elements are formed in stars; 
given data about fossils, reconstruct the possible environment in which 
the organisms lived; select geologic data to infer Earth’s tectonic plate 
boundaries; explain the factors that affect regional climates; use 
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Advanced 


knowledge of biogeochemical cycles to predict how an ecosystem may 
change due to pollutant or change in land use; and propose methods to 
lessen negative impacts on ecosystems. 


Students performing at the Advanced level should be able to use al- 
ternative models to generate predictions and explanations. They 
should be able to explain differences among physical, chemical, and 
nuclear changes; the wave and particle nature of light; paths of spe- 
cific elements through living systems; responses of ecosystems to 
disturbances; evidence for the theory of an expanding Universe; 
and evidence for human effects on the Earth’s biogeochemical cy- 
cles. They should be able to design and critique investigations that 
relate data to alternative models of phenomena. They should be able 
to compare costs or risks and benefits of alternative solutions to 
problems at local, regional, and global scales. 


Science Practices: Students performing at the Advanced level should be 
able to describe, measure, classify, explain, and predict phenomena at 
multiple scales, from atomic/molecular to cosmic; demonstrate relation- 
ships and use alternative models to generate predictions and explana- 
tions; design and critique observational and experimental studies, con- 
trolling multiple variables, making complex decisions about sampling, 
analyzing reliability of data, using scientific models to explain results, 
and choosing among alternative conclusions based on arguments from 
evidence; and compare scientific costs or risks and benefits of alterna- 
tive solutions to problems at local, regional, and global scales. 


In the physical sciences, students at the Advanced level should be able 
to describe how physical, chemical, and nuclear reactions differ; state 
the changes to a gas in a closed system with the addition of energy; sug- 
gest empirical evidence to demonstrate the conservation of matter in 
physical and chemical changes; describe energy transformations that 
occur in the transmission of electromagnetic waves and design an inves- 
tigation to identify the characteristics of electromagnetic waves; demon- 
strate the relationship of mass and velocity in conserving momentum 
during a two-body collision; analyze conflicting claims about scientific 
evidence related to issues such as effects of extended use of cell phones 
on the human brain and effective methods of containment of nuclear 
waste materials; and critique an experimental setup that measures veloc- 
ities of an object to obtain average acceleration. 


In the life sciences, students at the Advanced level should be able to 
predict changes in ecosystems in response to disturbances and trace el- 
ements through physical and chemical changes in cells, organisms, and 
ecosystems; analyze conflicting claims about scientific evidence related 
to biological issues such as genetically modified organisms and ecologi- 
cal effects of climate change; and design technological systems that mit- 
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igate harmful science-related effects on humans and ecosystems. 


In the Earth and space sciences, students at the Advanced level should 
be able to cite evidence (e.g., red shift) that the Universe expanded from 
a single point billions of years ago and that all but the lightest elements 
are formed in stars; use data from an excavation site to infer the age of a 
fossil; explain the mechanisms for phenomena at plate boundaries by 
employing earthquake data and using conceptual models; identify scien- 
tific trade-offs among energy sources; analyze conflicting claims about 
scientific evidence related to water resource issues such as ground water 
contamination and effects of stream channelization, levees, or dams on 
flood control and flood plains; and apply knowledge of biogeochemical 
cycles to predict changes that may occur if there is a disturbance in 
Earth’s systems due to a pollutant or the removal of a natural resource in 
an ecosystem. 
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APPENDIX C 
SAMPLE ITEMS AND SCORING GUIDES 


Item Source: TIMSS 1999, Grade 8 (framework, p. 65). 


Oak tree Caterpillar 


Look at the food web above. If the corn crop failed one year, what would most 
likely happen to the robin population? Explain your answer. 
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Note: A correct response must include a feasible explanation directly relating the predicted change 
in robin population to the effect of corn crop failure on prey/predator relationships indicated 
in the food web. Responses do not have to use the specific terms decrease, increase, and 
same, as long as the explanation is clear with respect to the effect on the robin population. If 
more than one effect is given, assign the code corresponding to the first correct explanation. 


Correct Response 


Robin population may decrease. Explanation based on predators (snakes/hawks) eating 
more robins if mice die. 
Examples: Goes down. The mice would starve and the snake would eat the robins. 
There would be less robins because the mouse population would 
decrease and the snakes (and/or hawks) would eat more of the robins. 


11 Robin population may increase. Explanation based on predators (snakes/hawks) dying 
due to lack of food (mice). 
Examples: It would go up because the snakes die if the mouse starves. 
70 
71 
72 
73 
79 


There could be more robins because there are fewer snakes (and/or 
hawks) to eat them. 


Robin population would stay the same with a feasible explanation. 
Example: — It would not change because the mouse would find other grain to eat 
so the snake would be unaffected. 


Other acceptable explanation. 


fea Incorrect Response 
Robin population would decrease. Incorrect explanation based on robins starving if 
snakes die (confuses prey/predator relationship). 
Examples: Decreases because there are less snakes to eat. 
When corn dies, then snakes, then robins starve. 


Robin population would decrease. Incorrect explanation based on the robin needing 
corn to survive. 
Example: — Decrease because they need the corn. 


Robin population would stay the same. Incorrect explanation based on the robins not 
needing corn to survive or not being connected to corn in the food web. (Does not 
consider the effect of predators.) 
Examples: — Nothing because the robin only eats insects. 
Nothing would happen. The corn is on a different chain in the food web. 


Mentions only that the whole food web will be upset and/or all the animals will die. 
Example: — The whole food web would erupt and everything would die. 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 
Ses Nonresponse 


Item Source: Colorado Department of Education 2002, Grade 8 (framework, p. 68). 
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The graph below shows the distance traveled over time by a student 
walking down a hall. Use the information shown on the graph to solve 
questions 7 and 8. 


Distance Traveled Over Time 


— ee NHN WwW 
nA omon ©& 


Distance (m) 
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WN 


10 15 20 25 30 
Time (seconds) 


7. During which time interval was the student moving the fastest? 
OA . 
OB 
O ¢ 
OD 
Key: D 
8. What was the average speed of the student from 0 seconds to 5 seconds? 


Average speed: 


Scoring: Average speed is 2 m/s. (The student traveled 10 meters in 5 seconds.) 


NAEP 2015 SCIENCE FRAMEWORK 
139 


Item Source: Quellmalz et al. 2004 (framework, p. 71). 


Lynx/Hare Task 


This is an interactive computer task in which students are expected to conduct a scientific 
investigation regarding the question of whether or not lynx should be introduced into a 
national park in order to reduce the abiding overpopulation of hares. Students are directed 
to complete six modules that use different computer programs to determine the best solu- 
tion for the proposed question: 


e Module 1 asks the student to access, organize, analyze, and interpret data they are 
given about the populations of hares over the past 4 years. For this module, 
students can use word processor, spreadsheet, or presentation software. 


e Module 2 asks the student to determine a better way to analyze and display some 
disorganized data that show how many lynx and hares were present each year 
over the past 25 years. 


e Module 3 first asks the student to submit a Web search that will give insight into 
the relationship between the lynx and hare populations. It subsequently asks the 
student to critically evaluate the relevance of several supplied Web searches. 


e Module 4 asks the student to collect information on specific questions regarding 
the lynx/hare question, to take notes on the information given on the websites, and 
to include citations for each site. 


e Module 5 asks the student to use a modeling program to predict the results of add- 
ing more lynx to the parks through viewing population trends over several years. 
Students are then asked questions based on what they have observed in the model- 
ing tool regarding increases or decreases in the hare population if lynx are added 
to the park or not added to the park. 


e Module 6 asks the student to create a presentation with word processor or presen- 
tation software to communicate the problem, the findings, and any recommenda- 


tions that resulted from the newly completed research. 


For this task, students receive scores for inquiry skills and technology use, and for the 
appropriate use of concepts within their explanations and recommendations. 


For more information, see http://ipat.sri.com/tasks/pred_prey/subtasks/taskstud.html. 
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Item Source: Washington Assessment of Student Learning 2004, Grade 8 (framework, p. 
75). 


Occasionally, a fire will destroy a forest, burning down trees and pushing wildlife out of their 
forest homes. However, the forest will grow back. Eventually, through the process of forest suc- 
cession as shown below, short grasses and flowers begin to grow and animals make new homes. 


Forest Succession 


Tall Shrubs trees 


Short grasses 
grasses Time 


Over time, shrubs and trees begin to grow. The forest returns to a lush habitat for the 
wildlife listed in the chart below. 


Forest Wildlife 
Ground dwelling Worms, beetles 
Reptiles and amphibians American toads, wood frogs, snakes, Eastern box turtles 
Small animals Squirrels, chipmunks 


Medium to large animals Opossums, raccoons, white-tailed deer, black bears 
Butterflies, moths, bees, wild turkeys, red-tailed hawks, 
bald eagles 


Airborne 


A power company owns part of a forest that was destroyed by a fire. The forest could 
take decades to rebuild on its own. The company’s department of environmental stud- 
ies suggests planting new trees to help the forest rebuild. 


Using the information in the scenario: 
e Explain how planting trees could benefit the natural ecosystem. 
e Explain how planting trees could harm the natural ecosystem. 
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Scoring Rubric 


2-point response: The response demonstrates that the student can analyze how human societies’ 
use of natural resources affects the quality of life and the health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the 
natural ecosystem. 

AND 

The student explains one reasonable way in which planting the trees could harm the nat- 
ural ecosystem. 


Example responses: 


Benefits of planting the trees include but are not limited to: 
e providing a habitat for animals; 
e providing a canopy, which would help prevent soil erosion; 
e creating root systems, which would anchor soil in place; and 
e creating shade, which would help maintain sunlight levels and inhibit the intro- 
duction of nonnative plant species. 


Harms of planting the trees include but are not limited to: 
e disrupting the natural flow of animals reentering the forest; 
e inhibiting the growth of other plants; 
e decreasing the diversity of tree species growing in the forest; and 
e introducing foreign species into an area, which may affect native species of 
plants and animals. 


1-point response: The response demonstrates that the student can partially analyze how human 
societies’ use of natural resources affects the quality of life and health of ecosystems. 


The student explains one reasonable way in which planting trees could benefit the 
environment. 

OR 

The student explains one reasonable way in which planting trees could harm the 
environment. 


0-point response: The response demonstrates that the student can perform little or no analysis of 
how human societies’ use of natural resources affects the quality of life and the health of 
ecosystems. 


Note: Benefits/harms to the natural ecosystem that only relate to humans will not be cred- 
ited score points. 
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Annotated example of a 2-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 


The ecosystem will rebuild more quickly, and that would give the 


animals a new habitat more quickly. 


Explanation of how planting trees could harm the natural ecosystem: 


It could disrupt the natural order of the ecosystem. If some 


smaller plants don’t get a chance to grow first, the trees might 


push them out. 


Annotation: 


The response demonstrates that the student can analyze how human societies’ use of natural re- 
sources affects the quality of life and health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the natural 
ecosystem, “The ecosystem will rebuild more quickly, and that would give the animals a new 
habitat more quickly.” (1 point) The student explains one reasonable way in which planting the 
trees could harm the natural ecosystem, “If some smaller plants don’t get a chance to grow first, 
the trees might push them out.” (1 point) 


Annotated example of a 1-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 


More plants and trees would mean giving off more oxygen, more shade, and 


more food for forest animals. 


Explanation of how planting trees could harm the natural ecosystem: 


There could be a very big storm and trees could fall on houses. 


Annotation: 


The response demonstrates that the student can partially analyze how human societies’ use of 
natural resources affects the quality of life and the health of ecosystems. 


The student explains one reasonable way in which planting the trees could benefit the natural 
ecosystem: “More food for forest animals.” (1 point) The student explains one way in which 
planting the trees could harm the human structure: “Trees falling on houses” but not the natural 
ecosystem. (0 points) 


NAEP 2015 SCIENCE FRAMEWORK 
143 


Annotated example of a 0-point response: 


Explanation of how planting trees could benefit the natural ecosystem: 


have more trees 


Explanation of how planting trees could harm the natural ecosystem: 


half the plants will die 


Annotation: 


The response demonstrates that the student can perform little to no analysis of how human socie- 
ties’ use of natural resources affects the quality of life and the health of ecosystems. 


The student states one factor, “have more trees,” but does not explain one reasonable way in 
which planting the trees could benefit the natural ecosystem. (0 points) 


The student states a possible harm, “half the plants will die,” but does not explain how planting 
the trees could cause this harm to the natural ecosystem. (0 points) 


Item Source: NAEP 1996, Grade 8 (framework, p. 83). 


A space station is to be located between Earth and the Moon at the place where Earth’s gravita- 
tional pull is equal to the Moon’s gravitational pull. On the diagram below, circle the letter indi- 
cating the approximate location of the space station. 


Earth 


Moon 


Explain your answer. 


Scoring Rationale: Student demonstrates the ability to explain the role of gravity in a manmade 
satellite and relates the force of gravity to the mass (size) of the object pulling it. 


3 = Complete - Student circles point C and gives a correct explanation that gravitational pull 
depends on mass and distance; thus, the station must be closer to the Moon because the 
Moon’s mass is less than that of Earth. 

2 = Partial - Student circles point C and explains that the Moon has less gravity than Earth 
but does not link it to mass. 

1 = Unsatisfactory/Incorrect - Student circles A, B, or C and gives an incorrect explanation 
or no explanation. 
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Sample Student Responses 
Complete (Level 3) 


Explain your answer. 


4 


AVEAAY Lb AA HMAAA PEAMMAL @ 4S thse LYLE 
station would have to located nearer to the 


moon to equal pulls 


Partial (Level 2) 


Explain your answer. 


Unsatisfactory/Incorrect (Level 1) 


Explain your answer. 


gravitational focus 
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Item Source: TIMSS 2003, Grade 4 (framework, p. 96). 


On a hot, humid day the air contains a lot of water vapor. What happens to the water vapor in the 
air when the air becomes very cold? 


Note: Priority should be given to code 10. If a response mentions condensation or freezing, 
code 10 should be the response even if other correct codes apply. Responses that mention 
ONLY that the water vapor becomes cold or rises without any mention of a change of 
state (explicitly or implicitly) are scored as incorrect (code 70 or 71). 


Correct Response 


2 
o 
oO 


Refers to condensation or freezing (or equivalent). 
Examples: _ It freezes. 

It condenses. 

Condensation. 

It condenses and turns into rain. 


Mentions cloud formation or a form of precipitation (e.g., rain, snow, fog, etc.). 
Examples: The water vapor changes to rain. 

It changes to snow. 

Water vapor turns into clouds. 

It rises into the clouds and becomes rain droplets. 

It turns foggy. 

It rains. 


Other correct 
Examples: It falls to the ground. 


Incorrect Response 


Mentions only that the water becomes cold. [No mention of a change of state or precipitation.] 
Examples: The water vapor becomes cold. 
Its temperature drops. 


Mentions only that water vapor rises (or similar). [No mention of condensation or precipitation.] 
Example: The water vapor will rise on a hot day. 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 
Example: It disappears. 


SI] Ni NI 
wo = oS 


Non-response 
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Item Source: New Standards Spring Field Test 1999 for High School (framework, p. 
96). 


The biosphere (living organisms), lithosphere (rocks and soils of Earth’s crust), and atmosphere 
are all involved in the cycling of carbon atoms. Describe the role each plays in the carbon cycle. 


Scoring guide: 


Score Point Description 

4 The response describes the cycling of carbon among all three reservoirs. The 
response is complete and detailed, showing evidence of logical reasoning. There 
is no evidence of misconceptions. 

3 The response describes the cycling of carbon between two of the reservoirs. The 
response may contain omissions or minor errors. 

2 The response describes the presence of carbon compounds in two or more 
reservoirs but may not link the reservoirs. The response may contain errors or 
misconceptions. 

1 The response is largely incomplete, lacks detail, and contains errors of fact and 
reasoning. 

0 The response may contain words from the question but does not add any 
information that might answer the question. 

Off topic Off-topic response. 

Blank No marks were made in the student response section. 


Background information for appropriate response: 


Carbon forms a variety of compounds. Carbon monoxide (CO) and carbon dioxide (CO;) are car- 
bon compounds that exist as gases in the atmosphere. Organic carbon compounds come in a wide 
variety of forms in living organisms: sugars, carbohydrates, cellulose, starches, collagen, chitin, 
amino acids, proteins, lipids, and nucleic acids. These organic compounds are also to be found in 
soil due to decaying and decomposition. The carbonate ion (CO,”) combines with different posi- 
tive ions to form carbonate compounds found in rocks and soil. 


Producers (green plants) take carbon dioxide from the air and convert it to sugar, using solar en- 
ergy (photosynthesis). Additional nutrients (including carbonate ions from the soil) are taken into 
the plant via its root system. These nutrients are rearranged and turned into the wide array of car- 
bon compounds listed above, along with the sugar compounds formed via photosynthesis. As 
consumers eat producers, the carbon compounds are broken down and rearranged into carbon 
compounds used by the consumer organism. 


When any living organism dies, the carbon compounds are broken down and returned to the soil 
by decomposer organisms. Huge sediment layers of dead marine organisms, which once had chi- 
tin cell walls or shells, compress under the weight of continuing sediments and the ocean. Uplift- 
ing of these layers forms sedimentary cliffs and rocks. Weathering and erosion continue to 
change the carbon compounds into forms that will again be incorporated into living organisms. 


Producer, consumer, and decomposer organisms harness the chemical bond energy stored in food 
via cellular respiration. Cellular respiration releases carbon dioxide back into the atmosphere. 


The score is based on evidence of: geochemical cycling of carbon. 
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Item Source: Shavelson, Baxter, and Pine 1991 (framework, pp. 102, 104). 


Electric Mysteries 


The following is a brief description of two warm-up tasks: 


1. Students are asked to connect one battery, one bulb, and wires so the bulb lights. They 
are then asked to draw a picture of this simple circuit. 
2. Given mystery box “?,” students are asked to identify whether it contains a battery or a 


wire. They are told they can determine the contents of the mystery box by connecting it 
in a circuit with a bulb. 


The following is an excerpt from the main task instructions given to students: 


Find out what is in the six mystery boxes A, B, C, D, E, and F. They have five different things 
inside, shown below. Two of the boxes will have the same thing. All of the others will have 
something different inside. 


[The five options—two batteries, a wire, a bulb, a battery and a bulb, nothing at all—are present- 
ed in words and drawings. Drawings are not provided here.] 


For each box, connect it in a circuit to help you figure out what is inside. You can use your bulbs, 
batteries, and wires in any way you like. 


When you find out what is in a box, fill in the spaces on the following pages. 


The following is an example of the student response format: 


Box A: HAS INSIDE. 


DRAW A PICTURE OF THE CIRCUIT THAT TOLD YOU WHAT WAS INSIDE BOX A: 


*[Al 


The following is a brief description of the scoring system: 


For each of the six boxes (A—F), students’ responses are scored on two components: (1) identifi- 
cation of the contents of the box, and (2) the circuit used to make the conclusion. For each box, if 
both components are correct, the student receives 1 point; if one or both components are incor- 
rect, the student receives 0 points. Total maximum score is 6 points. 
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Item Source: Persky et al. 2005 (framework, p. 103). 


TRE Web Search Task 


In this task, students are asked to use a search engine to determine why scientists use large helium 
gas balloons to explore outer space and the atmosphere instead of using satellites, rockets, or oth- 
er such tools. One open-ended question and several multiple-choice questions are presented to 
students. Students are scored on how well they performed the search, the quality of the book- 
marked pages, and how well the questions were answered. 


A complex scoring framework is used to assess students’ proficiency on the task. A “student 
model” is created that describes the “theory” of how different skills are linked together. For this 
task, there are five component skills: problem solving in technology-rich environments, computer 
skill, scientific inquiry skill, exploration, and synthesis. An “evidence model” is created that de- 
scribes how the students’ responses are connected to each of these skills. Once students begin the 
task, every action is recorded and connected to one or more skills in the student model. Then, a 
three-step process is used to evaluate this record. The first step is feature extraction, which shows 
what action the student took, when the action was taken, and any value that was associated with 
that action. The second step is feature evaluation, which gives the scores for the actions taken 
based on developed rules that show the best way to complete the task. The third step is evidence 
accumulation, which systematically combines responses into summary scores that detail the in- 
ferences that can be made from the students’ responses. 


This task incorporates a multifaceted evaluation process that can uncover the many skills in- 
volved in a single task or module. Thus, this module can provide a more comprehensive evalua- 
tion of students’ skills and aptitudes than traditional test questions. 


Item Source: Quellmalz et al. 2004 (framework, p. 105). 


Solar Power Task 


This is an interactive computer task in which students act as energy consultants to identify which 
two states will generate the most amount of electricity from photovoltaic cells. There are four 
modules in which students use a series of map visualizations and other data to reach and to pre- 
sent their final conclusions. Ultimately, students are asked to create a presentation to one state, 
recommending that the state apply for federal funds marked for solar energy use. 


Module 1 asks the student to review and apply background information on the conditions that 
both optimize and reduce solar energy production. They are also asked to conduct simple 
analysis using the ArcView map visualization program. 


Module 2 asks the student to explore several datasets to identify states with high incoming solar 
radiation and to manipulate the ArcView program and its map calculator tool. The student us- 
es the ArcView program to generate visualizations and to calculate which states will generate 
the best monthly and annual electricity yields from solar panels. 


Module 3 asks the student to determine what other data may be necessary to create the most com- 
pelling recommendation to the states. 
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Module 4 asks the student to create a presentation to one of the two states determined to have the 
highest capacity for solar energy production. The student uses the newly gathered data to 
support the recommendation to the state. 


Both generic and item-specific rubrics are used for scoring; they include scores for content (math 
and science), use of problem-solving or inquiry strategies (planning and thorough communica- 
tion), and use of technological tools. 


For more information, see http://ipat.sri.com/tasks/solarpower/subtasks/solar_tasks.html. 


Item Source: Adapted from Herl et al. 1999 (framework, p. 106). 
Concept-Mapping Task 


In this task, students use a custom software program to create a concept, or knowledge, map. Stu- 
dents are given 18 environmental science terms (atmosphere, bacteria, carbon dioxide, climate, 
consumer, decomposition, evaporation, food chain, greenhouse gases, nutrients, oceans, oxygen, 
photosynthesis, producer, respiration, sunlight, waste, and water cycle) and 7 link labels (causes, 
influences, part of, produces, requires, used for, and uses). Students can then drag and drop these 
concepts onto the grid space of the mapping program and add, erase, and link the items in their 
newly constructed maps. The concept maps are scored based on semantic content, organizational 
structure, number of terms used, and number of links made. 


Additionally, students explore a simulated World Wide Web space, which allows them to search 
for relevant environmental science information to improve their concept maps. Students can 
bookmark Web pages they believe are helpful in constructing their concept maps. This portion of 
the task is scored based on relevant information found, the hypertext links that were selected 
(browsing), keyword searching, and the accessing of three or more highly relevant Web pages for 
a single concept (focused browsing). 


While performing the concept-mapping task, students are able to access real-time feedback, 
which compares students’ maps to experts’ maps and gives corresponding feedback about which 
items are correct and which need improvement. A score is also assigned to how often a student 
accessed feedback, a measure of monitoring one’s learning. 


This computer-based assessment provides a detailed view into both the student’s ultimate perfor- 
mance and the steps of the thought processes that were employed to generate the ultimate product 
or answer (in this case, the concept map). 
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APPENDIX D 
GROUP 2 SMALL-SCALE SPECIAL STUDIES 


KNOWING WHAT STUDENTS KNOW ABOUT TECHNOLOGICAL DESIGN 


Knowledge about technology and the technological design process are prominent in both 
the National Standards and Benchmarks. The National Standards state, “Although these 
are science education standards, the relationship between science and technology is so 
close that any presentation of science without developing an understanding of technology 
would portray an inaccurate picture of science” (NRC 1996, p. 190). Despite taking some 
time for schools to include technology in the curriculum for all students, there is growing 
recognition that technology should be an important component. An increasing number of 
states include technology in their standards—30 states in 2001 and 38 states by 2004 
(Meade and Dugger 2004). Consequently, NAEP is including the process of technologi- 
cal design as parallel to, but with less emphasis than, the process of scientific inquiry. 


Because relatively few questions on NAEP will probe the practice of Using Technologi- 
cal Design, this study proposes the development of an additional set of questions to probe 
students’ understanding of this practice in-depth. 


Specifically, this study would address the following research question: 


What do students know about technological design in the contexts of agricultural 
technologies, energy generation technologies, and technologies related to Earth mate- 
rials and resources? 


This study was not conducted because the Governing Board added a new assessment of 
Technology and Engineering Literacy in 2014 that covers this content. 


EXTENDED INVESTIGATIONS BY STUDENTS 


National and local science education standards emphasize scientific inquiry. In many 
states, this goal requires student engagement in projects that can take days, weeks, and 
even months as they undertake genuine investigations. Important outcomes of these pro- 
jects include a range of skills that are a crucial feature of high-quality science education 
but that cannot be assessed adequately in a 50-minute assessment (NRC 2005). They in- 
clude, for example, gauging the quality of students’ (a) reasoning while framing their re- 
search questions; (b) planning for data collection and the execution of that plan; (c) abil- 
ity to meet unpredictable challenges that arise during any actual, ongoing scientific inves- 
tigation; (d) persistence in seeking productive explanations for their observations and re- 
vising plans for the investigation; (e) lines of argument in deciding how to alter their ex- 
perimental approach in the light of new evidence; (f) engagement with fellow students 
and/or the teacher in interpreting an observation or result and deciding what to do about 
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it; and (g) deliberations when settling on the defensible conclusions that might be drawn 
from their work. 


In many countries, teachers are expected to assess students’ work during extended pro- 
jects. Often their judgments of student achievement are made during ongoing classroom 
activities that are part of the regular curriculum. The teachers’ assessments are incorpo- 
rated into an overall score that also includes results of the short, timed tests. In some 
countries, a defined percentage of the total score is based on teachers’ judgments about 
achievement associated with investigative projects. 


This study, then, might include both a national sample of students and an exploration of 
what other countries do under similar circumstances. Specifically, the study would ad- 


dress the following research questions: 


e What methods can be or have been developed to assess student achievement with 
respect to the ability to conduct extended scientific investigations? 


e To what extent are shorter investigations interchangeable for the extended inves- 
tigation and to what extent are they not interchangeable? 


This study was not conducted because of lack of available funding. 
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